-- Logs begin at Tue 2022-08-30 00:08:57 UTC, end at Tue 2022-08-30 01:15:24 UTC. -- Aug 30 00:41:19.118318 dv-cinder-815746-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 00:41:19.126599 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 00:41:19.128702 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 00:41:19 2022] *** Aug 30 00:41:19.128702 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 00:41:19.128702 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 00:41:19.128702 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: nodename: dv-cinder-815746-8-iscsi Aug 30 00:41:19.128702 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: machine: x86_64 Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: clock source: unix Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: pcre jit disabled Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: detected number of CPU cores: 4 Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: current working directory: / Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: detected binary path: /usr/bin/uwsgi-core Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: your processes number limit is 47680 Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: your memory page size is 4096 bytes Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: detected max file descriptor number: 2048 Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: lock engine: pthread robust mutexes Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: thunder lock: enabled Aug 30 00:41:19.129105 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 00:41:19.129675 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 00:41:19.169409 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: Python main interpreter initialized at 0x55f8774b19c0 Aug 30 00:41:19.169409 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: python threads support enabled Aug 30 00:41:19.169409 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: your server socket listen backlog is limited to 100 connections Aug 30 00:41:19.169409 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: your mercy for graceful operations on workers is 90 seconds Aug 30 00:41:19.169848 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 00:41:19.169848 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: *** Operational MODE: preforking *** Aug 30 00:41:19.170152 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: *** uWSGI is running in multiple interpreter mode *** Aug 30 00:41:19.170152 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: spawned uWSGI master process (pid: 105773) Aug 30 00:41:19.170365 dv-cinder-815746-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 00:41:19.171439 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: spawned uWSGI worker 1 (pid: 105787, cores: 1) Aug 30 00:41:19.171439 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: spawned uWSGI worker 2 (pid: 105788, cores: 1) Aug 30 00:41:19.171439 dv-cinder-815746-8-iscsi devstack@keystone.service[105773]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 00:41:23.446584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:41:23.446584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: from cryptography.utils import int_from_bytes Aug 30 00:41:23.447372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:41:23.447372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: from cryptography.utils import int_from_bytes Aug 30 00:41:23.447490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:41:23.447490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: from cryptography.utils import int_from_bytes Aug 30 00:41:23.447490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:41:23.447490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: from cryptography.utils import int_from_bytes Aug 30 00:41:23.496538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:41:23.496718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:41:23.496911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] command line args: [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:41:23.497178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:41:23.497322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:41:23.497322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ================================================================================ {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:41:23.497457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:41:23.497529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] command line args: [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:41:23.497587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] admin_token = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.497803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:41:23.497942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] config_dir = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ================================================================================ {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:41:23.498088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] config_source = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] admin_token = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] debug = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] config_dir = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] config_source = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] debug = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.498988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] insecure_debug = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] insecure_debug = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.499902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_config_append = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_dir = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_file = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_config_append = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_dir = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.500965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_file = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.501904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.502962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] notification_format = cadf {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.503919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] notification_format = cadf {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] publish_errors = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.504907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] publish_errors = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.505771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] standard_threads = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] strict_password_check = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] standard_threads = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] strict_password_check = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] use_eventlog = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] use_journal = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.506970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] use_json = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] use_eventlog = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] use_stderr = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] use_journal = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] use_syslog = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] use_json = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] use_stderr = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] watch_log_file = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] use_syslog = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.507876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] watch_log_file = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:41:23.507953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.508928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.509839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.510937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.511923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.512961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.513993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.514938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.515923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.516999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.517062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.517430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.517571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.518822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.518822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.518822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.519452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.519452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.519632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.520125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.520429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.520708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.520992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.521288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.521534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.521776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.522020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.522261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.522502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.522736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.522989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.523252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.523495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.523732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.523996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.524236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.524691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.525138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.525722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.526361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.526938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.527536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.527814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.527950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.enabled = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.528970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.529986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.530946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.531968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.532935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.533966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.534878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.535809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.external = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.536895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.mapped = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.password = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.537924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] auth.token = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] catalog.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.538966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.539945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.540881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.enabled = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.541922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.542831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.543912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.544951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.545990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.546877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.547951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.548912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.549864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.550956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.551984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.552918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.external = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.mapped = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.553979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.password = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.554837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] auth.token = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] catalog.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.555986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.password = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.556975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.557878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.558931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.559964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.560967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.561831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.562929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.563972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.564916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.565889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] policy.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.566855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] receipt.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.567922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.568993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.569886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.570865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] revoke.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] role.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.571959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] role.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] role.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] role.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.572964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.573946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.574899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.575770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.576891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.577915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.578957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.579922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.password = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.580839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.provider = fernet {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.581944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.582130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.582255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.582388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.582648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.582796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] trust.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.583861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.584875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.connection = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.585929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.586980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.587952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.588960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.589829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.590061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.590222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105788) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:41:23.590441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.590812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.591204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.591573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.591944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.592277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.592518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.592770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.593010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.593247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.593485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.593723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.593968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.594215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.594450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.594708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] policy.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.594940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.595214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.595453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.595688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] receipt.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.595927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.596175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.596417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.596650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.596888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.597231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.597504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.597754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.597994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.598243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.598494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.598731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] revoke.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.599313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.599550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.599791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] role.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.600031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] role.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.600276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] role.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.600507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] role.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.600756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.601004 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.601251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.601499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.601742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.601988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.602234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.602477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.602717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.602964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.603235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.603483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.603728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.604161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.604404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.604655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.604906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.605164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.605541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.605785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.606041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.606293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.606533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.606767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.607017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.607251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.607503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.607759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.608019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.608240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.608475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.608706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.608942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.609179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.609426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.provider = fernet {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.609662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.609919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.610166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.610406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.610651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.610897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.611169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] trust.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.611407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.611647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.611896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.612126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.612409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.612647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.612942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.613259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.613527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.connection = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.613784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.614017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.614259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.614523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.614760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.614997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.615256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.615494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.615741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.615977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.616222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.616458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.616706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.616941 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.617190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.617423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.617663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.617902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:41:23.618063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105787) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:41:23.620192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.620428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.620655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.629055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.642944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.643203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.643429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.651963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.656656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.656893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.670326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.670460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.710243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.714664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.758664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.758848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.760114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.760356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.917470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.917683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.917926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.918141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.918396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.918589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.934038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.934316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.934625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.934951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.935299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.935621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.943134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.943371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.943598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.943824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.944042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.944248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.944537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.954706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.955044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.955502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.955625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.955828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.956151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.956487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.956882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.967956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.968950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.969188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.969423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.969616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.969857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.970080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.970296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105788) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.972453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.972825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.973200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.973456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.973665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.974190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.974467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105787) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:41:23.989345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.989534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:41:23.989908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.990102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:41:23.990382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.990642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.990894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.991173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.994742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.995097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:41:23.995579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.995854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:41:23.996227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.996597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.996898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:23.997153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.000782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.001064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.001337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.001621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.007153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.007436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.007707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.007993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.009171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.012110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.012413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.012680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.012967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.013249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.013508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.013775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.014045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.014313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.014589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.014855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.015152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.018056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.018357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.018618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.018901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.019342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.019611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.019867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.020142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.020418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.020693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.020962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.038695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.040201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.040488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.044165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.045038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.045322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.045602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.045725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.045911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.050635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.051135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.051422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.052189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.052474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.056441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.056736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.057019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.057301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.057585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.057907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.058195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.059671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.059955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.060238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.062214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.062505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.062795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.063210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.063487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.063762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.064036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.064328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.075947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.076228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.080152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.081885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.082171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.083390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.083692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.086075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.088068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.089223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.089643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.093900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.094327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.094630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.094915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.095220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.095510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.095891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.097080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.100696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.100998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.101403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.101712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.101994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.102260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.112292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.112587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.112899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.113151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.118463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.118723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.119042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.119276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.120030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.121689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.123274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.123552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.126200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.126324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.126489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.126781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.127072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.127382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.127924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.129778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.130053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.132740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.133031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.133321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.133607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.133905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.138862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.139179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.139454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.139732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.140008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.140287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.140580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.145541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.145840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.146129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.146414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.146699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.147065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.147351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.156306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.159384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.162724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.164196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.164899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.165679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.165960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.166250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.167965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.170877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.172359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.172797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.173853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.174140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.174312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.174416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.177254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.177544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.177831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.178104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.181150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.182706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.185779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.186023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.186113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.186316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.186410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.186599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.186687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.193608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=105788) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.193894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.194156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.194433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.194707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.194821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.194976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.195160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.195245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.195432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.195515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.195801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.196087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.196356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.196622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.196882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.197164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=105788) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.202365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=105787) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:41:24.202652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.202934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.203231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.203519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.203779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.204059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.204328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.204608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.204877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.205147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.205418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.205678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.205949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=105787) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:41:24.231871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f8774b19c0 pid: 105788 (default app) Aug 30 00:41:24.235496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc0ca-212a-49f6-8e8c-217e38551026 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:24.235832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc0ca-212a-49f6-8e8c-217e38551026 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:24.236152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc0ca-212a-49f6-8e8c-217e38551026 None None] PATH_INFO: `/v3` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:24.240430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Tue Aug 30 00:41:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:24.241254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f8774b19c0 pid: 105787 (default app) Aug 30 00:41:30.854097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:30.854952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:30.854952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:30.857122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:30.865789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:30.866366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:30.866366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:30.936169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: INFO dbcounter [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] Registered counter for database keystone Aug 30 00:41:30.957699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG oslo_db.sqlalchemy.engines [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105787) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:41:30.969018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writer thread running {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:41:31.164679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG passlib.handlers.bcrypt [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=105787) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:41:31.164953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG passlib.handlers.bcrypt [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=105787) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:41:31.224204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:31.225831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:31.382547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:41:31.866403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:31.941908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: INFO dbcounter [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] Registered counter for database keystone Aug 30 00:41:31.967090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG oslo_db.sqlalchemy.engines [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105788) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:41:31.977798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writer thread running {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:41:32.106435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdb5d-d924-42fd-8453-0fe84442b31b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:32.115438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:32.116528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:32.116895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:32.117347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:32.132707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:32.185515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:41:32.185856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc821a08-1466-429d-ac5e-8a3d107d9c51 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:32.198445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Aug 30 00:41:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:41:32.207870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:32.223185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1134f2-57dc-4222-8948-e8aefdd87e32 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:32.228818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:32.229929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:32.230206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:32.230475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] PATH_INFO: `/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:32.232050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] RBAC: Authorizing `identity:get_project(project_id=aa5e798ada384a4f94fc4eb7a83dd47f)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:32.284746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:41:32.285336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b48b13-aba6-4928-987a-886f93a2c6fc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:32.289950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Aug 30 00:41:32 2022] GET /identity/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:34.185913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf916c-b1e5-49c9-bdcb-582efb167b5d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:34.186810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf916c-b1e5-49c9-bdcb-582efb167b5d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:34.186810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf916c-b1e5-49c9-bdcb-582efb167b5d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:34.192639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:34.199409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:34.200119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:34.200285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:34.230768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:34.232362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:34.242098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:41:34.504413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf916c-b1e5-49c9-bdcb-582efb167b5d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:34.518148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf916c-b1e5-49c9-bdcb-582efb167b5d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:34.524165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:34.524757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:34.525032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:34.525268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:34.533670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:34.535684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5547d0-ef80-438d-8bdb-89e27f6dc64a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:34.544387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Aug 30 00:41:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:41:34.550436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:34.566084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3542b-ec34-46cf-ba97-2660cbe005b8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:34.572765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:34.573401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:34.573682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:34.573940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:34.575510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:34.577423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12841293-08bf-4c2c-92d5-4cba747b1dc0 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:34.578972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Tue Aug 30 00:41:34 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:36.561826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86223244-28c5-43cb-9813-9ee0816f25bd None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:36.562391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86223244-28c5-43cb-9813-9ee0816f25bd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:36.562490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86223244-28c5-43cb-9813-9ee0816f25bd None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:36.563351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44dee-19e8-4034-9689-5bceef658659 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:36.563734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44dee-19e8-4034-9689-5bceef658659 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:36.563971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44dee-19e8-4034-9689-5bceef658659 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:36.564346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:36.566042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 5/11] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:36.572703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:36.573184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:36.573639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:36.574547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:36.574965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:36.575498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:36.596662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:36.598883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:36.619774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG passlib.handlers.bcrypt [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=105788) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:41:36.620181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG passlib.handlers.bcrypt [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=105788) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:41:36.670029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:36.671596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:36.683991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:36.776143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:36.964279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:36.991664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.044592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.075875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d8e47-0e41-47f8-beb8-210e2c235e63 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.110646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.111295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.111578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.111856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.147861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdf763-ee9a-4356-a49c-4f122768a796 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.152360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.192973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.193902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.194461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.195050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.195325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.195599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.207717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:37.216406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:37.217357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:41:37.218258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 8/14] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:37.220044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.224132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.298821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 7/15] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:41:37.304504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.326263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8055-2fcf-49b6-9874-60f1322b4f84 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.360408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.361089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.361379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.361651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.362402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:37.362942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.364425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.376495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 9/16] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:37.382751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.407172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e89ae-7801-455b-a163-e1549b4ebb75 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.413828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.433138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.446369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.447032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.447314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.447584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.458284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.459830 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:41:37.478453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:37.479558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:41:37.480374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:41:37.480374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:41:37.480374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:37.480374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:41:37.485766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.490689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ead8d-767c-4630-9f24-320f1c5f7998 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.525211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.525853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.526130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.526402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.527925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.529418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.531023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:37.536975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.589876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af4119-ed4b-4abf-b1a4-eae4f80417ce None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.626152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.626775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.627085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.627359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.628063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:37.628529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.629952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.632245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.639738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4f57d-8bc8-4868-a609-4265b0141bcb None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.649115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:37.678184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.678878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.679197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.679470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.680664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.682156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2187c-15ed-4143-9c41-6f3689e8ec94 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.683932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:37.690445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:37.731055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:37.801277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf9e1-efcd-4f35-8216-62f50cf82398 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:37.835178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:37.835794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:37.836055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:37.836322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:37.837051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.domain.id=default, user.id=65d191ebc1aa4d0ebd90e4982cc77782) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:37.837350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:37.838815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f31c1f-83bc-4aea-950c-816416d4d183 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:37.845635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Aug 30 00:41:37 2022] GET /identity/v3/role_assignments?user.id=65d191ebc1aa4d0ebd90e4982cc77782&scope.domain.id=default&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:39.807153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722002-d9cc-44d4-a0d6-61f3476732c9 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:39.807646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722002-d9cc-44d4-a0d6-61f3476732c9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:39.807646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722002-d9cc-44d4-a0d6-61f3476732c9 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:39.809370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:39.814653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0a9f-6a41-40b8-9d42-77d1be335f60 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:39.815120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0a9f-6a41-40b8-9d42-77d1be335f60 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:39.815330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0a9f-6a41-40b8-9d42-77d1be335f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:39.844035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-217d0a9f-6a41-40b8-9d42-77d1be335f60 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:39.845240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-217d0a9f-6a41-40b8-9d42-77d1be335f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:39.941321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:40.100379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992aa-d31f-4aff-a9bb-07ff11434d0d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.100696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992aa-d31f-4aff-a9bb-07ff11434d0d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.100931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f992aa-d31f-4aff-a9bb-07ff11434d0d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.102268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 13/24] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:40.107672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaaf72-7c59-492a-99ec-f2aa0cdef9dd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.107955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaaf72-7c59-492a-99ec-f2aa0cdef9dd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.108225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaaf72-7c59-492a-99ec-f2aa0cdef9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.136004 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c5aaaf72-7c59-492a-99ec-f2aa0cdef9dd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:40.137202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c5aaaf72-7c59-492a-99ec-f2aa0cdef9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:40.165353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:40.213973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 12/25] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:40.315145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaf9ed-47e6-4b22-a7f4-2267845d5d21 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.315612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaf9ed-47e6-4b22-a7f4-2267845d5d21 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.315832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaf9ed-47e6-4b22-a7f4-2267845d5d21 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.317822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:40.323676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.323962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.324238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.345252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:40.347041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:40.448765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 13/27] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:40.532963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaf9ed-47e6-4b22-a7f4-2267845d5d21 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:40.575534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:40.697919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaf9ed-47e6-4b22-a7f4-2267845d5d21 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:40.735654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:40.736299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.736576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.736847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.747625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:40.760833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:40.762383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-16fb2adf-aeec-47eb-9b24-4880275d94be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:41:40.782243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:40.783186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:41:40.784159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:41:40.784159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:41:40.784159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:41:40.784763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 15/28] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:40.785976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06cc3-9d0c-4fbe-9571-0a9fb71ded56 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.786414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06cc3-9d0c-4fbe-9571-0a9fb71ded56 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.786699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06cc3-9d0c-4fbe-9571-0a9fb71ded56 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.792841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 16/29] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:40.793688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06cc3-9d0c-4fbe-9571-0a9fb71ded56 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:40.796988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7981fd-05f5-47d9-bf47-75032ba683bc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:40.811837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:40.877118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:40.877820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.878384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.879186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.879932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:40.881443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:40.917553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06cc3-9d0c-4fbe-9571-0a9fb71ded56 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:40.954852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:40.955404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.955682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.956040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:40.956885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:40.957186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:40.958583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9655f50-1af0-4314-963d-6aa0f76bab2a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:40.980466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 17/30] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:40.981910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:40.982348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:40.982726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.004493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:41.005780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:41.010686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 14/31] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 30 00:41:40 2022] POST /identity/v3/roles => generated 225 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:41:41.012010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.089609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 18/32] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:41.090494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.099454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:41.189868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee08c9c-875a-4316-b463-94ee1c53611f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:41.271662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:41.272219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.272559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.272836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.273531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:41.275136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:41.306248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 15/33] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 30 00:41:41 2022] POST /identity/v3/roles => generated 219 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:41:41.312825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95754e-9d73-4879-8ca2-2693e460d965 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:41.359183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:41.360027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.360381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.360690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.361361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:41.363327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783321-9791-40b2-bbcb-038b0da5155e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:41.403486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.419681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:41.537885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e470-8b65-429a-a4d4-d8fc14a001b4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:41.554531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 19/34] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:41:41 2022] POST /identity/v3/projects => generated 301 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:41:41.556368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aca07-41a9-4b69-9074-ab3bc839964b None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.556664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aca07-41a9-4b69-9074-ab3bc839964b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.556949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aca07-41a9-4b69-9074-ab3bc839964b None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.559206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 20/35] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:41.565682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8fea-4faf-4d01-b3ee-19ff8d8d1835 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.565899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8fea-4faf-4d01-b3ee-19ff8d8d1835 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.566218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e8fea-4faf-4d01-b3ee-19ff8d8d1835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.578969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:41.580034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.580534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.581388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.588549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-226e8fea-4faf-4d01-b3ee-19ff8d8d1835 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:41.590151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-226e8fea-4faf-4d01-b3ee-19ff8d8d1835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:41.596894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:41.604351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:41.605266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:41.606037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:41.608461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 16/36] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:41:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:41.615766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.665962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:41.690497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 21/37] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:41.714077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:41.726649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbe4b-aac6-45ab-8c18-53c357befabd None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.726966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbe4b-aac6-45ab-8c18-53c357befabd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.727236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbe4b-aac6-45ab-8c18-53c357befabd None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.729001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 22/38] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:41:41.734556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.734868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.735214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.757978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:41.759642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:41.765890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6d34e-ed97-4a42-ae44-cde3a67348cc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:41.802933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:41.803263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.803497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.803866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.804463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:41.804765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:41.806175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:41.819713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 17/39] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:41.824784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.844697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 23/40] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:41.934307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3dbe-82c9-45bd-852b-3f6af766bd5a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:41.965840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:41.974485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:41.978419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:41.979142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:41.979480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:41.979744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:41.980948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:41.982447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:41.984264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 18/41] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:41 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:41.989860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:42.083905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798ba56-96ec-4939-ab94-5a2f71cbc5cb None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:42.103418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:42.123517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:42.124148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:42.124431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:42.124708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:42.125398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:42.126953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:42.146142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 24/42] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 30 00:41:41 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:41:42.146820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:42.222171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9713718-7ea5-44dd-a3dc-b35092726caa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:42.261144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:42.261808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:42.262092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:42.262373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:42.264263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:42.266041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:42.273409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 19/43] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:42.274856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:42.277257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942ea92-36f1-4152-9716-badf14860f1e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:42.323699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:42.324367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:42.324657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:42.324938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:42.326166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:42.327732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:42.330245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 25/44] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:42.337384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:42.367367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:42.399489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:42.403591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f9fac-c7fb-40e5-a4f0-6bb1a43f8c8e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:42.437800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:42.438474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:42.438755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:42.439058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] PATH_INFO: `/v3/domains/default/users/65d191ebc1aa4d0ebd90e4982cc77782/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:42.441091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=65d191ebc1aa4d0ebd90e4982cc77782, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:42.442556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097669b2-156f-4e68-8073-5d9693178e2f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:42.457028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 20/45] 10.222.0.17 () {58 vars in 1361 bytes} [Tue Aug 30 00:41:42 2022] PUT /identity/v3/domains/default/users/65d191ebc1aa4d0ebd90e4982cc77782/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:41:42.458728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda589-efae-4555-85c2-3b35c934a6cd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:42.500001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:42.500635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:42.500914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:42.501185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:42.501859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:42.503657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f089f-cb90-42dc-af07-2f06adf4e809 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:42.537164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 26/46] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:41:42 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:41:44.330136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdae3-9380-44c0-8a52-8e906fc633d0 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:44.330719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdae3-9380-44c0-8a52-8e906fc633d0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:44.330719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdae3-9380-44c0-8a52-8e906fc633d0 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:44.332262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 21/47] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:44.337830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390f93e-2cec-4aba-9b01-ad6861b77d26 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:44.338284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390f93e-2cec-4aba-9b01-ad6861b77d26 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:44.338391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390f93e-2cec-4aba-9b01-ad6861b77d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:44.359658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1390f93e-2cec-4aba-9b01-ad6861b77d26 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:44.360852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1390f93e-2cec-4aba-9b01-ad6861b77d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:44.431693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 27/48] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:44.435809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6711ab-f6a0-43bb-b4ad-cf7667a7367e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:44.436199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6711ab-f6a0-43bb-b4ad-cf7667a7367e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:44.436541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6711ab-f6a0-43bb-b4ad-cf7667a7367e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:44.437920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 22/49] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:44.443842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:44.444139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:44.444418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:44.464380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:44.465635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:44.487325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:44.538351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 28/50] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:44.699091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6711ab-f6a0-43bb-b4ad-cf7667a7367e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:44.800764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:44.806839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6711ab-f6a0-43bb-b4ad-cf7667a7367e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:44.834692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:44.845475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:44.846142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:44.846640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:44.847026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:44.860411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:44.865802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:44.866760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:44.867551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 23/51] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:44.874024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:44.985105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab57841-0185-4c23-8594-3e51fdbfabfa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.024151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.025072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.025447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.025718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.026420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:45.026585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28850ad6-61fa-4d83-a0fb-b3796ef246fb None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.026690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.028202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.038791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 24/52] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:45.045913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:45.073137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.074080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.074448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.074790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.076525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.078709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.081558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 29/53] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:45.086787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:45.105804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:45.113434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:45.157536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360c5a0-94a8-476d-9bdc-f3c960579d5a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.188485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d21f9-1be7-42f9-86de-35cf15b0337f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.192693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.193343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.193624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.193893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.195068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.196541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.202776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:45 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:45.203799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:45.224939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.225583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.225851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.226125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.226775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.228264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.255835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 30/55] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Aug 30 00:41:45 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:41:45.307680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9918d2-103a-494c-a7e8-a5db488b9534 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.328018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:45.349035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.350454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.350792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.351165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.352523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.354371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8492aaf-0db4-4c9e-9c66-4f811fd81711 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.356062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 26/56] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:45.362226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:45.446285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:45.469483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95200-262b-4d0b-a55f-10312a3ec0a7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:45.504506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:45.505134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:45.505419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:45.505693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:45.506497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.domain.id=default, user.id=65d191ebc1aa4d0ebd90e4982cc77782) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:45.506767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:45.508254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61af64e-1b12-4ae7-9a89-19f97bb5a98a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:45.517649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 31/57] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Aug 30 00:41:45 2022] GET /identity/v3/role_assignments?user.id=65d191ebc1aa4d0ebd90e4982cc77782&scope.domain.id=default&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:47.183873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c3cd-d92f-4728-9265-6dc38b98c106 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:47.184777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c3cd-d92f-4728-9265-6dc38b98c106 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:47.184777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c3cd-d92f-4728-9265-6dc38b98c106 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:47.186955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 27/58] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:47.194444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50153-6680-4874-8f58-388332b2d11c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:47.195137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50153-6680-4874-8f58-388332b2d11c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:47.195302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50153-6680-4874-8f58-388332b2d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:47.224789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-37d50153-6680-4874-8f58-388332b2d11c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:47.226010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-37d50153-6680-4874-8f58-388332b2d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:47.307586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 32/59] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:47.578148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c3cd-d92f-4728-9265-6dc38b98c106 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:47.688964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c3cd-d92f-4728-9265-6dc38b98c106 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:47.724875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:47.725504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:47.725784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:47.726058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:47.727233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:47.728700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0f7ba-ed7e-4b6f-8230-c84e998e8aed None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:47.730276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 28/60] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:47.736221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d50153-6680-4874-8f58-388332b2d11c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:47.811316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:47.845688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d50153-6680-4874-8f58-388332b2d11c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:47.882013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:47.882653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:47.882935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:47.883242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:47.883936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:47.885415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c568331-dbcf-4803-96dd-3a4517fbe807 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:47.915845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 33/61] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:41:47 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:41:50.695067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71681df-7559-4831-a6ec-46f0a77693cc None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:50.695558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71681df-7559-4831-a6ec-46f0a77693cc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:50.695840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71681df-7559-4831-a6ec-46f0a77693cc None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:50.697236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 29/62] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:50.702486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae944-4ff5-4e57-9039-b026f80bccc7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:50.702888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae944-4ff5-4e57-9039-b026f80bccc7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:50.703074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae944-4ff5-4e57-9039-b026f80bccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:50.736495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c57ae944-4ff5-4e57-9039-b026f80bccc7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:50.738407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c57ae944-4ff5-4e57-9039-b026f80bccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:50.835180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 34/63] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:51.140208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71681df-7559-4831-a6ec-46f0a77693cc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:51.212270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:51.279993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71681df-7559-4831-a6ec-46f0a77693cc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:51.306666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228f1ce-887b-4f5c-8726-e7a4b102c83a None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.306975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228f1ce-887b-4f5c-8726-e7a4b102c83a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.307248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228f1ce-887b-4f5c-8726-e7a4b102c83a None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.309387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 35/64] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:41:51.314864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe2473-1f46-4da7-87d5-e31076c42979 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.315183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe2473-1f46-4da7-87d5-e31076c42979 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.315403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe2473-1f46-4da7-87d5-e31076c42979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.331899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:51.332869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.333356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.333836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.345358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a1fe2473-1f46-4da7-87d5-e31076c42979 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:51.348729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a1fe2473-1f46-4da7-87d5-e31076c42979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:51.351552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:51.356632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:51.357472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:51.358307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:51.360859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 30/65] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:51.371181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:51.475732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:51.478393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563c845-ecfa-4495-a961-e35466611a22 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:51.493559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:51.513451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 36/66] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:51.516698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:51.517532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.518160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.518764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.519979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:51.520949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:51.522593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:51.532044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 31/67] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:41:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:51.535168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039868f1-4b5e-4137-975c-7baf224ec450 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.535670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039868f1-4b5e-4137-975c-7baf224ec450 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.536156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039868f1-4b5e-4137-975c-7baf224ec450 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.537695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 37/68] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:51.538648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:51.548447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.549093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.549646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.580081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:51.581770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:51.660391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a3fec-1983-42da-862e-9f3e4ebe6c6b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:51.660853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 38/69] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:51.696250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:51.696825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.697274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.697696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.713588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:51.715397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:51.717648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 32/70] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:51 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:51.724160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:51.828320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431758ef-0a27-4008-b1d5-f852438c2943 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:51.845666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:51.865639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:51.866559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:51.866987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:51.867427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:51.873050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:51.944133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:51.972866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:51.974599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995c4fe-4f56-4846-a310-ca681e4e8e2f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.027725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed602029-f450-4299-b71a-4fc42815f940 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.066885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.067831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.068118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.068383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.078602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 39/71] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:41:52.079349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:52.079951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ca1f-2506-4c2c-ba20-7d914eefba8d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.080542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ca1f-2506-4c2c-ba20-7d914eefba8d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.080948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ca1f-2506-4c2c-ba20-7d914eefba8d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:52.085173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:52.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:52.086864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 33/72] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:41:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:52.087540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 40/73] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:52.087540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ca1f-2506-4c2c-ba20-7d914eefba8d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.093055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.107307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.190723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.191293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ca1f-2506-4c2c-ba20-7d914eefba8d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.199693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f554-7562-459e-9384-a25c75e42f99 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.228236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.228808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.229089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.229365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.230606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.232156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33364176-9edd-4ff6-8229-d3193a25d90c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.233815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 41/74] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.234612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.234936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982aa1c-c10e-4683-9374-2b99cd9d2b9e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.235248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982aa1c-c10e-4683-9374-2b99cd9d2b9e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.235439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.235534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982aa1c-c10e-4683-9374-2b99cd9d2b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.235683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.235916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.247034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:52.251904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:52.253003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:52.253761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 34/75] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:52.254411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.255808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5982aa1c-c10e-4683-9374-2b99cd9d2b9e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:52.257057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5982aa1c-c10e-4683-9374-2b99cd9d2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:52.329797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 42/76] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:52.331102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fc5bd-3f96-4ebc-bea8-d257b842ebec None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.331394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fc5bd-3f96-4ebc-bea8-d257b842ebec None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.331672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fc5bd-3f96-4ebc-bea8-d257b842ebec None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.332948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 43/77] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:52.333645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fc5bd-3f96-4ebc-bea8-d257b842ebec None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.351572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.365215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c488d-27a6-4d19-9a20-f6862858f1eb None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.407153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.407802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.408095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.408670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.409300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:52.409695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.412878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.426978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 35/78] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.426978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.441894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.451525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fc5bd-3f96-4ebc-bea8-d257b842ebec None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.487656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.488207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.488502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.488809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.489525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:52.489830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.491327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98100ab0-9870-4c5f-b781-040b85cb1592 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.497074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 44/79] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.498518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.498974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.499488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.520271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:52.521601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:52.553598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efdcf-50da-4d84-835e-f03046c79415 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.598642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.599812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.600105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.600387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.601112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:52.601409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.602918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.614946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 36/80] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.615985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.623788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 45/81] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:52.624700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.640851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.711473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.742134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5b16-208c-4a9d-9a09-8d2a379ecb10 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.766515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713d111-5622-40c7-95e8-fa913e23fc66 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.780118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.780916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.781326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.781710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.783018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.785453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.787323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 37/82] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.788235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.812484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.813619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.814042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.815086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.817009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.819514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.821288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 46/83] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.822824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.908789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc13e9-6546-4cd8-a3ea-91e8107f9ee2 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.931426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:52.953202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:52.953926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:52.954305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:52.954580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:52.956044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:52.957524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:52.958577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390177-0e95-4215-bc2b-c4f06c51ac4e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:52.963771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 38/84] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:52.963771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:52.979507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.000438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.000990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.001272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.001551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.074980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.076655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.077344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de330-1433-4c89-be37-b9a636adcb2c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.117334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.117959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.118245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.118521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.119235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.120795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.149692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 39/85] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:41:52 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:41:53.150544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.172432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.211740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 47/86] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:41:53.213754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.235222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.278466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a97b-3567-4a0b-b8d5-647b42c0ca3f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.322106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.322975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.323482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.323974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.338680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:53.345609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:53.346630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:53.347880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:53.348476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 40/87] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:53.349531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.358859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1f7b0-f3d1-4666-8273-fbb83108952f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.404541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.405179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.405902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.406183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.466626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.496455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89a04-ba12-41ad-a4e6-a3ffb10834e5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.508771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.510795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.541127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.542354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.543105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.543346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.543759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.544931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.546424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.548157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 41/88] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:53.549083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.629782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 48/89] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:41:53.630667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.674295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83546db-c22c-49fe-8540-ad0f0011a838 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.715918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.716602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.716840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.717121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.717796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:53.718105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.719980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.730689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 42/90] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:53.736816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.815460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.855761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3860d-a8f0-468d-91fc-d22a2a9ffcbb None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:53.864985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:53.895664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:53.896496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:53.896806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:53.897089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:53.898395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:53.899993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:53.901920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 43/91] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:53.903271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:53.926985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4702c-14c9-4107-ab92-131769419292 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:54.001315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:54.002382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.002839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.003296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.004205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:54.004715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:54.006649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:54.021978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 49/92] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 259 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:54.023244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:54.037137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22089-9213-45e1-9fc8-3469f2274c0d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:54.123411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:54.163743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:54.164787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.165265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.165729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.167372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:54.169059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:54.171355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 44/93] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:53 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:54.179126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:54.252110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3a6b7-9209-4065-9769-74d2299e92b9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:54.325757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:54.326413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.326700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.326993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.327785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=65d191ebc1aa4d0ebd90e4982cc77782) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:54.328108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:54.330546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:54.345982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 50/94] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:41:54 2022] GET /identity/v3/role_assignments?user.id=65d191ebc1aa4d0ebd90e4982cc77782&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:54.380992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5fb98-6921-495c-9786-a9d28c0d3f6f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:54.432672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:54.433319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.433606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.433882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.526246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:54.534185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:54.535945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060557-0b13-4635-bca7-c50ad53f36b1 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:54.672371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 45/95] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:41:54.678317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:54.682346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:54.791946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851000-6b2e-4c14-a070-3b2443ad6c30 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.792524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851000-6b2e-4c14-a070-3b2443ad6c30 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.792524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851000-6b2e-4c14-a070-3b2443ad6c30 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.794065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 46/96] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:54.804837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.805266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.805651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.806996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba010a9-d7bb-477b-a558-a90dcc0b8036 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:54.848717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:54.852295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:54.854176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:54.855120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:54.857733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:54.858162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] PATH_INFO: `/v3/projects/c43022d5c4ef425bbcec09499db12d8c` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:54.859546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] RBAC: Authorizing `identity:get_project(project_id=c43022d5c4ef425bbcec09499db12d8c)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:54.862066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:54.865772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:54 2022] GET /identity/v3/projects/c43022d5c4ef425bbcec09499db12d8c => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:54.872119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:54.965595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 47/98] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:54.984996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:55.025509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0765c-136f-45ff-90f1-fade210abe5a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:55.071729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:55.072359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.072645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.072920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.073662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c43022d5c4ef425bbcec09499db12d8c, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:55.073975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:55.075491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:55.082236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 52/99] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:41:54 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=c43022d5c4ef425bbcec09499db12d8c&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:55.378258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:55.418359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:55.497880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6ba23-5015-45e3-8e61-1468ab2164ad None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:55.532770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:55.533620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.533902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.534175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.544777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:55.549884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:55.550775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:55.551607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 48/100] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:55.558538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:55.684808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aabfc3-ef5d-496d-9495-b0a3c00b5f93 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:55.721357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:55.722336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.722678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.722955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.723648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:55.723954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:55.725343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd828e8-2547-4c28-883e-dcc2e3746f97 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:55.727899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:55.737122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 53/101] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:41:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:55.742973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:55.781517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0d1f-d363-4907-bd49-cb85f0ef3e80 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.781832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0d1f-d363-4907-bd49-cb85f0ef3e80 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.782082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c0d1f-d363-4907-bd49-cb85f0ef3e80 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.783734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 54/102] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:41:55.788901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.789215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.789501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.807795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:55.808982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:55.832816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:55.849172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660e930-5356-4c48-9348-7f92d7a45f15 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:55.918582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:55.919409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:55.919693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:55.919972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:55.922277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:55.924044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:55.925958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 49/103] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:55 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:55.931691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:55.949106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 55/104] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:56.071955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f5e7e-1604-49b3-9444-a9f7a9ceb112 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:56.108136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:56.108797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.109087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.109359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.110517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:56.112042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dda27-beb4-42bc-995a-c7089aab143f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:56.114085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 50/105] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:55 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:56.119847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:56.181963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f76a5-76ec-4600-b05a-64976b02c07e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.182405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f76a5-76ec-4600-b05a-64976b02c07e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.182794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f76a5-76ec-4600-b05a-64976b02c07e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.184615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 51/106] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:41:56.190293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.190578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.190919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.215546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:56.216691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:56.248290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:56.288839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce341f-7c80-4cc5-b0e0-616e8dcace0f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:56.347631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:56.356920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 52/107] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:41:56.367090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:56.367777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.368067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.368350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/13fa28a8fed841f499f8ca013dd715e7/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.373445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, user_id=13fa28a8fed841f499f8ca013dd715e7, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:56.374916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:56.398741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:56.417205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 56/108] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:41:56 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/13fa28a8fed841f499f8ca013dd715e7/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:41:56.547971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef96b-f4d5-4ca2-9ce5-f4cd07f58693 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:56.598634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:56.599428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.599720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.599998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.614863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:56.623965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:56.624835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:56.625583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 53/109] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:41:56.631581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:56.784793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d9a38-b793-4ee4-be52-9261eae3df8b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:56.808061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:56.824649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:56.858682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:56.859369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:56.859656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:56.859939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:56.860659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:56.861186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:56.865863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:56.884176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 57/110] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:41:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:56.889050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:56.999514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:57.021204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0743-6b88-4c86-96c9-dfca3bb2b865 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.113015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:57.113679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.113954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.114218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.119732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:57.121193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07b063-2696-417a-8825-0f172119b896 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.121325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:57.122841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 54/111] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:41:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:57.129294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:57.216456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:57.217069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.217543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.217960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] PATH_INFO: `/v3/users/ff71d71f10dc41f5b3159b205c2da035` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.258067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] RBAC: Authorizing `identity:get_user(user_id=ff71d71f10dc41f5b3159b205c2da035)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:57.260216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:57.262776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 58/112] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:56 2022] GET /identity/v3/users/ff71d71f10dc41f5b3159b205c2da035 => generated 289 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:57.272069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:57.272892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:57.324975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64aba1-06d2-4699-bdb6-c16364a97546 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.372830 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:57.373773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.374270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.374743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.375625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:57.377352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c87c8-f3e8-4233-bce1-857985ef4670 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:57.387745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d44aa-31ac-4261-9c06-9a3555351bdf None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.402823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:57.416155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 55/113] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:41:57 2022] POST /identity/v3/projects => generated 302 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:41:57.417809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2d525-d4db-455a-97ed-402a0b13cdde None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.418305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2d525-d4db-455a-97ed-402a0b13cdde None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.418780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2d525-d4db-455a-97ed-402a0b13cdde None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.420787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 56/114] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:57.422356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210957b4-f420-4ba6-a65f-280ab8b42447 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.422851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210957b4-f420-4ba6-a65f-280ab8b42447 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.423380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210957b4-f420-4ba6-a65f-280ab8b42447 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.425471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 57/115] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:57.427170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6a3e-a1cc-440e-b4f4-6d55d9b3bcb1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.427798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6a3e-a1cc-440e-b4f4-6d55d9b3bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.428266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6a3e-a1cc-440e-b4f4-6d55d9b3bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.437216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:57.437868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.438158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.438434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.452562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-799e6a3e-a1cc-440e-b4f4-6d55d9b3bcb1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:57.454338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-799e6a3e-a1cc-440e-b4f4-6d55d9b3bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:57.519524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:57.521115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:57.542884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 58/116] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:57.545606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.546487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.547204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.575494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:57.577607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:57.638474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 59/117] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:41:57.644772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:57.645925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:57.661928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:57.670649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 59/118] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:57.761236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fad6-db4b-453f-8b5e-5abf5fffd14d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.796681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:57.797331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:57.797615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:57.797896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:57.799474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:57.801080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:57.802982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 60/119] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:57 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:57.808757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:57.947558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddac2d-0c8c-4575-9ecd-619a4d5817a2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:57.996125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.003772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.004706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.005206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.005683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.006751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=ff71d71f10dc41f5b3159b205c2da035) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:58.007285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:58.008974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:58.018509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:41:57 2022] GET /identity/v3/role_assignments?user.id=ff71d71f10dc41f5b3159b205c2da035&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:58.034423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.095777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:58.129109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57222564-dd61-4622-860b-7e199e624e8f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.146480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:58.180627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.183505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.184920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.185398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.211902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:58.224980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:58.225901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:41:58.226669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:41:58.232613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 61/121] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:41:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:58.239903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77590763-bf89-42d7-9938-3d5d3f3dcbd9 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.242853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.314074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.314792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.315114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.315451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.330737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-48990686-bf0d-4d49-b565-2f54759bf781 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:58.335837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:58.336755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:41:58.337592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 61/122] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:41:58.338857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b42-c8b4-49b5-a8e0-26caaccd631a None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.338857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b42-c8b4-49b5-a8e0-26caaccd631a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.339112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03b42-c8b4-49b5-a8e0-26caaccd631a None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.340484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 62/123] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:58.344389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03b42-c8b4-49b5-a8e0-26caaccd631a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.399677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a67d2-089e-4503-9ece-f75b07266cae None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.473536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.474137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.474459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.474679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.475455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:58.475766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:58.477329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9002a-64ee-4883-8a30-4c52ee1aa08d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:58.487341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:58.497746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 62/124] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:58.498640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03b42-c8b4-49b5-a8e0-26caaccd631a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.503874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.509244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.509776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.527383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:58.543955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:58.545239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:58.569076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.570091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.570561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.571047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.571911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:58.572399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:58.573999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:58.594609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 63/125] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:58.595335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.676560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 63/126] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:58.677363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:58.836137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded381-ece6-43ef-a333-a48fcb9f1553 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.888745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:58.893203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:58.893865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:58.894193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:58.894471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:58.896100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef796960-2e82-4eb6-bedb-fa9f7bb5fecd None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:58.903057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:58.903057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:58.903312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 64/127] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:58.909618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.034845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710877b2-d770-44eb-b7c8-132ffc558e9c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.047723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.049089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.049566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.050059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.051538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.053234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.055301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 64/128] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:59.061064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.092857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.093442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.093724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.094001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.096045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.100149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.100261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 65/129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:58 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:41:59.101173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.177329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:59.199431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:59.245772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587200f0-fa18-41d0-9804-abbb2309f39d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.261899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dbacb-63aa-4fd8-b25f-66e83de1d299 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.284952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.285678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.285954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.286232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.287481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.288986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.290843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 65/130] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:59 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:59.293071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.335723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.336862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.337406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.337876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.349402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:41:59.354969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:41:59.355839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:41:59.356588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:41:59.358792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 66/131] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:41:59.359780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.432180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc8c8b-1639-4e72-a448-fea603a19442 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.447773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:59.473625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.474421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.474768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.475383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/13fa28a8fed841f499f8ca013dd715e7/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.478339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, user_id=13fa28a8fed841f499f8ca013dd715e7, role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.480104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bef30-7c61-4472-b5bf-e78bd4444972 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.488293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b931d9-b991-4c19-8ca6-cc55716c957b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.500127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 66/132] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:41:59 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/13fa28a8fed841f499f8ca013dd715e7/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:41:59.502358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06734-fdd3-4c17-9a18-c0bac3cd9695 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.502965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06734-fdd3-4c17-9a18-c0bac3cd9695 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.503552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06734-fdd3-4c17-9a18-c0bac3cd9695 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.505375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 67/133] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:41:59.506490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06734-fdd3-4c17-9a18-c0bac3cd9695 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.539420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:59.563792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.564565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.564840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.565116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/65d191ebc1aa4d0ebd90e4982cc77782/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.572011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, user_id=65d191ebc1aa4d0ebd90e4982cc77782, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.573540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f5263-80c5-4ad8-8be3-04d2027afe4e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.605304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 67/134] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:41:59 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/65d191ebc1aa4d0ebd90e4982cc77782/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:41:59.606434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.606790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.607091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.646471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:41:59.647739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:41:59.697108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06734-fdd3-4c17-9a18-c0bac3cd9695 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.766732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.767419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.767698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.768271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.768643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:41:59.769079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.770533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.781273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 68/135] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:41:59.782005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 68/136] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:59.786052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:41:59.904972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff005e-8fb5-4535-86f9-356479e0754f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:41:59.921247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:41:59.944722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:41:59.945540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:41:59.945833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:41:59.946105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:41:59.947406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:41:59.948957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:41:59.950708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 69/137] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:41:59 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:41:59.956759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.012816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:00.097077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c2ef7-1d83-4e0f-af8d-0b13b8cd9374 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:00.137641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:00.138308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.138624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.138874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] PATH_INFO: `/v3/projects/c43022d5c4ef425bbcec09499db12d8c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.140067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] RBAC: Authorizing `identity:get_project(project_id=c43022d5c4ef425bbcec09499db12d8c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:00.141533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:00.143456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 69/138] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:41:59 2022] GET /identity/v3/projects/c43022d5c4ef425bbcec09499db12d8c => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:00.149494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.276905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.282992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc658-3f4d-4596-8547-1c5344f6e048 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:00.319633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:00.320680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.321167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.321674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] PATH_INFO: `/v3/projects/c43022d5c4ef425bbcec09499db12d8c/users/13fa28a8fed841f499f8ca013dd715e7/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.323875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] RBAC: Authorizing `identity:create_grant(project_id=c43022d5c4ef425bbcec09499db12d8c, user_id=13fa28a8fed841f499f8ca013dd715e7, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:00.325724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:00.342529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 70/139] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:00 2022] PUT /identity/v3/projects/c43022d5c4ef425bbcec09499db12d8c/users/13fa28a8fed841f499f8ca013dd715e7/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:00.420579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a135-8d04-4151-95bd-c60710dc584b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:00.422182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:00.459641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:00.460322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.460582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.460852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.473127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b3517d00-3101-4e0c-9e6f-79a047da367f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:00.478574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:00.479516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:00.480250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:00.484638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 70/140] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:00.488140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.497061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f042c-3c56-412c-b3b3-8691e89dfcba None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.497705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f042c-3c56-412c-b3b3-8691e89dfcba None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.498163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f042c-3c56-412c-b3b3-8691e89dfcba None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.500299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 71/141] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:00.517446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde064ee-06c0-4fe5-a45e-6e6f294acc39 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.518253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde064ee-06c0-4fe5-a45e-6e6f294acc39 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.518727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde064ee-06c0-4fe5-a45e-6e6f294acc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.549356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fde064ee-06c0-4fe5-a45e-6e6f294acc39 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:00.551443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fde064ee-06c0-4fe5-a45e-6e6f294acc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:00.592401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:00.651868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 72/142] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:00.653852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f65da9-492f-4099-aedb-fe2593a2549d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.654327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f65da9-492f-4099-aedb-fe2593a2549d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.654783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f65da9-492f-4099-aedb-fe2593a2549d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.656877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 73/143] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:00.662530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.663157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.663731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.680804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89275e-9bbb-4a20-a791-09f9bb38de31 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:00.683772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:00.685163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:00.726585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:00.727263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.727543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.727820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.728710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:00.729023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:00.731351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:00.742713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 71/144] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:00.749163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.778751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 74/145] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:00.861401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad72c-5636-4984-80ef-3b3418ac5d79 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:00.863955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:00.896248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:00.896893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:00.897175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:00.897451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:00.898606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:00.900119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:00.901711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 72/146] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:00 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:00.907614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:00.949818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:01.039533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.059617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a36a82-8c53-414c-a157-b01338862e67 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:01.121237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:01.121845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:01.122067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:01.122304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:01.152333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be8818-4d4b-4565-bb82-ec6692bd9904 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:01.191849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:01.192501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:01.192783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:01.193063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:01.194201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:01.196244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:01.197855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 73/147] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:01.204259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.251517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:01.268192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:01.269712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:01.323497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:01.348447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f269c-d8e8-4635-8520-e4ada5c39272 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:01.422065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:01.423140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:01.423659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:01.424357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:01.425286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:01.427113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:01.457271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 74/148] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:01 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:01.458756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.529846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 75/149] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:01.540002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.591142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc9da8-206b-4d9a-a67b-6dc1d7f7a98f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:01.595480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:01.683837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:01.684413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:01.684695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:01.684970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:01.701255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:01.706452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:01.707541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:01.708482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:01.710060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 75/150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:01.716415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.801061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce830-5384-452c-8f8a-a4fb77083542 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:01.843460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:01.845965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:01.847129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:01.847415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:01.847693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:01.849891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:01.851520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:01.853421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 76/151] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:01 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:01.859292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:01.932334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96226d43-edc3-4f13-96f7-f5e942eeaa41 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:02.000875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:02.001532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.001745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.002027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.002805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:02.003136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:02.004788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:02.007960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e165aba-8a0e-41f2-b970-8c18ef7f492c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:02.025181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 76/152] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:02.031600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:02.067964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:02.080728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:02.081357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.081632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.081900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.082644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:02.082951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:02.085605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:02.092879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 77/153] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:01 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:02.206588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385a45-bf65-437f-9ed8-7167e8b24029 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:02.249883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:02.250528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.250809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.251165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] PATH_INFO: `/v3/users/ff71d71f10dc41f5b3159b205c2da035` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.255961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] RBAC: Authorizing `identity:get_user(user_id=ff71d71f10dc41f5b3159b205c2da035)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:02.257452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b259c-3f6b-4fa8-ad77-01e183912e2b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:02.259191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 77/154] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:02 2022] GET /identity/v3/users/ff71d71f10dc41f5b3159b205c2da035 => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:02.264956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:02.338174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:02.384562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea281ff-6848-45f0-b423-767758b5054b None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.384879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea281ff-6848-45f0-b423-767758b5054b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.385123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea281ff-6848-45f0-b423-767758b5054b None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.386477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 78/155] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:02.392313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851b64-ca75-43f6-aa92-a8f7478dc329 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.392589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851b64-ca75-43f6-aa92-a8f7478dc329 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.392863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851b64-ca75-43f6-aa92-a8f7478dc329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.406632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788a7c-b621-4387-94b4-cf198dc89963 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:02.431944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0b851b64-ca75-43f6-aa92-a8f7478dc329 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:02.433165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0b851b64-ca75-43f6-aa92-a8f7478dc329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:02.478891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:02.479982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.480263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.480535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.481707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:02.483676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:02.485412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 78/156] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:02 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:02.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:02.560744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 79/157] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:02.611342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc025803-c20b-4e07-b7fb-894b56faa312 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.611943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc025803-c20b-4e07-b7fb-894b56faa312 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.612299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc025803-c20b-4e07-b7fb-894b56faa312 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.613987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 80/158] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:02.620595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.620954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.621401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.629712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7810645-4ac7-4788-8369-e1022f544dd0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:02.652169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:02.653499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:02.659151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:02.678205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:02.686626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:02.687731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:02.688158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:02.688591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/ff71d71f10dc41f5b3159b205c2da035/roles/2ef85303f3c64cd59d98007e7782d03b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:02.691134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, user_id=ff71d71f10dc41f5b3159b205c2da035, role_id=2ef85303f3c64cd59d98007e7782d03b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:02.693027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:02.716081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 79/159] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:02 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/users/ff71d71f10dc41f5b3159b205c2da035/roles/2ef85303f3c64cd59d98007e7782d03b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:02.790493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 81/160] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:03.007669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.147066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53734790-c72e-48fa-983c-dbe0dac32757 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:03.198397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:03.199067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.199350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.199619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.216383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:03.222460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:03.223453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:03.224843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 80/161] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:03.229627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.300131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.361998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8221e6-6e01-4140-804e-9383a6d53d87 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:03.378870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:03.422238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:03.422857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.423176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.423606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.424282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:03.424591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:03.425986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:03.438274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 82/162] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:03.461286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.484067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:03.519506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d77e8-eb5d-4a19-a60f-b046f4ac2b98 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:03.595469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:03.595469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.595469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.603446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.625879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-28520fb9-afd4-46ed-bb77-8b1af731a5ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:03.636957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:03.637859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:03.639077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 81/163] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:03.639956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95cd5-e5a6-45bb-82c5-6b299ca62837 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.639956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95cd5-e5a6-45bb-82c5-6b299ca62837 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.639956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95cd5-e5a6-45bb-82c5-6b299ca62837 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.641175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 82/164] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:03.648965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c95cd5-e5a6-45bb-82c5-6b299ca62837 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.664606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05cf24-c133-4143-83ff-773c80d54665 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:03.727767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:03.728427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.728706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.728975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.730733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:03.732229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3c558-44f4-410e-98ab-374c68f2a9c2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:03.736240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 83/165] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:03.736240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.736240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.736240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.767212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:03.768775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:03.868946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 84/166] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:03.869983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:03.881187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:03.911732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c95cd5-e5a6-45bb-82c5-6b299ca62837 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:03.913776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:03.973665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:03.974310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:03.974586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:03.974855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:03.975575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:03.975873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:03.977332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:03.998201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 83/167] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:04.004360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:04.038204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae995b-3566-4059-b262-a0c230893c99 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:04.098233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:04.098887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:04.099199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:04.099473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:04.120542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc6aa9-680f-4fe8-a8fd-cfa8c2c10036 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:04.163246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:04.163921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:04.164192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:04.164454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] PATH_INFO: `/v3/users/65d191ebc1aa4d0ebd90e4982cc77782` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:04.165737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] RBAC: Authorizing `identity:get_user(user_id=65d191ebc1aa4d0ebd90e4982cc77782)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:04.167275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:04.170133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 84/168] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:04 2022] GET /identity/v3/users/65d191ebc1aa4d0ebd90e4982cc77782 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:04.181151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:04.239564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:04.241050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:04.247397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:04.259794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:04.360020 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e4fe-8ab6-472e-b3ce-526628017d3c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:04.446275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:04.447471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:04.447753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:04.448031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:04.531636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 85/169] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:04.532500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:04.609524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:04.611937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:04.676946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:04.813531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 85/170] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:04.814981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:04.817863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436578-55fe-44fe-a918-cc41996a6f3a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:04.829802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:04.897090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:04.897740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:04.898018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:04.898288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:04.932418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:04.936757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:04.938109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 86/171] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:04.939661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.063107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7695fd5-0492-4d32-9238-cdd2728a4f8d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.121327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e7642-1057-45b4-a55d-de4364b4cee2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.122419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:05.135738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.136421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.136710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.136973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.138200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.139731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.145322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 86/172] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:04 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.145322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.182761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.183918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.184954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.185441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.188063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.190070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.192744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 87/173] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:04 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.193767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.249234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:05.373084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a0e0-3936-4ffe-ac33-b3dadb64e371 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.407286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6d87-079f-4f5f-921b-c6cda4c2784c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.428328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.428985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.429248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.429522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.430277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:05.430586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.432095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.448278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 88/174] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:05 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.449132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.500986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.501664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.501952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.502201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.502889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:05.503220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.504634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d73e22-70eb-4e14-980a-f267ebc2fce3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.521317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 87/175] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.522792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da795a-b190-4ed9-896a-197b3e04f4e3 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.523114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da795a-b190-4ed9-896a-197b3e04f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.523389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da795a-b190-4ed9-896a-197b3e04f4e3 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.524946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 88/176] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:05.525899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb544b-8605-4e7a-b1b1-65013ca5c887 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.526239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb544b-8605-4e7a-b1b1-65013ca5c887 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.526509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb544b-8605-4e7a-b1b1-65013ca5c887 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.533681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 89/177] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:05.534567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb544b-8605-4e7a-b1b1-65013ca5c887 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.572271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9051eaa-e90d-4ba0-83cd-c1c6e40aeb40 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.574227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:05.613157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.613901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.614163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.614437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.615212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=65d191ebc1aa4d0ebd90e4982cc77782) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:05.615523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.617012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979471a-1524-4ef3-b7c7-fc4d6970b2e4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.623680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 89/178] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:05 2022] GET /identity/v3/role_assignments?user.id=65d191ebc1aa4d0ebd90e4982cc77782&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.625183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.625471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.625742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.648086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:05.649425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:05.701388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:05.752507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb544b-8605-4e7a-b1b1-65013ca5c887 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.794220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.794905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.795206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.795479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] PATH_INFO: `/v3/users/13fa28a8fed841f499f8ca013dd715e7` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.796641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] RBAC: Authorizing `identity:get_user(user_id=13fa28a8fed841f499f8ca013dd715e7)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:05.798271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32c1ab-322b-42b0-b289-8770cc142725 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:05.800583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 90/179] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:05 2022] GET /identity/v3/users/13fa28a8fed841f499f8ca013dd715e7 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:05.801771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 90/180] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:05.801888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e0572-5250-409f-80b5-a322b87a1b8c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.802067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e0572-5250-409f-80b5-a322b87a1b8c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.802384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e0572-5250-409f-80b5-a322b87a1b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.807574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:05.834910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-af7e0572-5250-409f-80b5-a322b87a1b8c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:05.839823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-af7e0572-5250-409f-80b5-a322b87a1b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:05.944433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:05.955207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d204322-d809-43aa-b713-42fd24f9f007 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:05.970222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 91/181] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:05.990367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86a38c-e041-401c-a4cb-bde687487f7f None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.990653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86a38c-e041-401c-a4cb-bde687487f7f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.990928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86a38c-e041-401c-a4cb-bde687487f7f None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.992503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 92/182] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:05.995850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:05.996682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.997126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.997578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:05.999119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c560-c9ab-4558-ab64-3a69214421ee None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:05.999400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c560-c9ab-4558-ab64-3a69214421ee None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:05.999670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c560-c9ab-4558-ab64-3a69214421ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.027712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a022c560-c9ab-4558-ab64-3a69214421ee None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:06.028914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a022c560-c9ab-4558-ab64-3a69214421ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:06.104123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.118228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 93/183] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:06.134233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.135764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.169819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702440e1-d390-400f-8552-3e30b718a02c None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.170794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702440e1-d390-400f-8552-3e30b718a02c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.171263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702440e1-d390-400f-8552-3e30b718a02c None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.172857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 94/184] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:06.183046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.183939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.184700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.208998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:06.210529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:06.280441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.293924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 91/185] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:06.294702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.296465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 95/186] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:06.299857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.415264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75930dd4-7bd2-4b72-bb64-432f0cdeb068 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.422885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f7bdf-91a7-4853-b74a-54ee9e870803 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.446696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.453242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.453875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.454152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.454676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.462166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.462741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.463109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.463330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] PATH_INFO: `/v3/projects/c43022d5c4ef425bbcec09499db12d8c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.464571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] RBAC: Authorizing `identity:get_project(project_id=c43022d5c4ef425bbcec09499db12d8c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.465275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:06.465987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.471140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 96/187] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/projects/c43022d5c4ef425bbcec09499db12d8c => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:06.471140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:06.471637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:06.472481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:06.473252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 92/188] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:06.477083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.535903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.581734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1109-99ea-4382-8493-044c3aa0c2ac None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.584743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a9e53-8d15-407e-966a-771a7ffed91f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.615510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.616146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.616417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.616674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.618792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.619743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.620025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.620300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.626938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:06.630645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:06.631750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:06.632651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:06.633419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 97/189] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:06.634071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:06.635742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:06.636597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 93/190] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:06.639764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.688274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.738688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9940a8-ed1a-4f1d-8f22-ffaa1340e9c1 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.740203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a4416-7057-44c9-b5ee-327b9f267ff8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.753649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.772114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.772733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.773009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.773279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.774036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c43022d5c4ef425bbcec09499db12d8c, user.id=13fa28a8fed841f499f8ca013dd715e7) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:06.774342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.775432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.775838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.776054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.776320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.776588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.777240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:06.777697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.779116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.782377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 98/191] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/role_assignments?user.id=13fa28a8fed841f499f8ca013dd715e7&scope.project.id=c43022d5c4ef425bbcec09499db12d8c&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:06.782377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.792490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 94/192] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:06.793168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.912276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1048-b464-478b-81c6-7949c38e94f4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.915685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9778d3e-0250-4f22-ade6-4997c5064380 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:06.926812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:06.952326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.953091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.953374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.953649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.955159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:06.955302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.955786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:06.956063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:06.956334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:06.956722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.958025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:06.958025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:06.958324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 99/193] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:06.958634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:06.959605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:06.968477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 95/194] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:06.969147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.016198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.074475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6bbe-8507-4a8e-b9b9-2a746a89f3bb None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.080838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d51b8f-3a37-476a-8516-7592d33fe0bd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.107873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.108484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.108757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.109022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.109683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:07.109975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.111429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.114948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.115596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.115869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.116137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.119486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 100/195] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:07.120246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.130512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.132079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.133729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 96/196] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:06 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:07.134486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.166218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.222783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98856b-120d-44f0-a296-d9b55921e874 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.230020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.237608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5cbb-661f-4f0f-b3e8-819efb3899e8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.258240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.258893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.259199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.259466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.260121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.261595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.270263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.270859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.271167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.271439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.272874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.274819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.277054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 97/197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:07.277756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.282480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 101/198] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:07 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:07.284102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.400023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05b461-7311-44ea-9dfb-7c8aca43954b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.408915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.413157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5da68-8902-4b07-b383-c56a0c9d3071 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.452475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.453148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.453425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.453694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] PATH_INFO: `/v3/users/ff71d71f10dc41f5b3159b205c2da035` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.455082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] RBAC: Authorizing `identity:get_user(user_id=ff71d71f10dc41f5b3159b205c2da035)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.456519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.457135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.457775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.458275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.458377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 98/199] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/users/ff71d71f10dc41f5b3159b205c2da035 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:07.458450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.459628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.499666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.547432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.549386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.589798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cfad-b516-4ed6-a9a8-976b733995a1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.634504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.635747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.636197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.636778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.691247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 102/200] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:07.692236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.696372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.715039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.716661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.749218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.826274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 99/201] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:07.827103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:07.839275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf3452-fe90-4a07-a804-6fceeb25f06f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.875992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.876671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.876982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.877357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.943427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a404ae-510d-4392-a19a-e1a223f881ec None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:07.948374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.952012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.953948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.977444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:07.990113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:07.990804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:07.991513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:07.991788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:07.993497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:07.995685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:07.997623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 100/202] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:07.999500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:08.090606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 103/203] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:08.091462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:08.125915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa931b7-51f2-4b8c-b008-ff0c573ac097 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:08.169584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:08.170492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:08.170771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:08.171081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:08.172215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:08.174599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:08.177118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 101/204] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:07 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:08.177970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:08.214778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:08.220621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0e7ad-fdca-447d-9617-dc2ea89520d7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:08.263359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:08.263690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:08.263973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:08.264253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:08.264741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:08.265481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=0dccc6c01cb14cde8a0a7f8ed68c7297) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:08.266510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:08.268110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:08.275275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 104/205] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:08 2022] GET /identity/v3/role_assignments?user.id=0dccc6c01cb14cde8a0a7f8ed68c7297&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:08.276088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:08.323922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95378bac-d39b-4c14-bbda-0efff9c98ba3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:08.408301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:08.409093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:08.409314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:08.409592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:08.413908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:08.415585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:08.417193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 102/206] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:08 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:08.425057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:08.472551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45666e1d-6601-4ccd-b503-73be376b991f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:08.525826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:08.526468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:08.526644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:08.531050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:08.531050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=0dccc6c01cb14cde8a0a7f8ed68c7297) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:08.531050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:08.531050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2b80c-2b29-4ac3-a641-f23589ff3d6b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:08.545042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 105/207] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:08 2022] GET /identity/v3/role_assignments?user.id=0dccc6c01cb14cde8a0a7f8ed68c7297&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:08.587032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a965b6-bd42-46be-ad1f-865c47c6d2ac None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:08.630673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:08.631343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:08.631626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:08.631904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:08.632667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, scope.project.id=a462dd813727475a9e0d05c4c7c465bc, user.id=ff71d71f10dc41f5b3159b205c2da035) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:08.632980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:08.634498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05346a4f-5076-4332-8f53-348c79a415ec None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:08.636648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:08.641955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 103/208] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:08 2022] GET /identity/v3/role_assignments?user.id=ff71d71f10dc41f5b3159b205c2da035&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:09.552109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86c25-a204-4563-a6b6-bf50a54f415a None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:09.552866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86c25-a204-4563-a6b6-bf50a54f415a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:09.552866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86c25-a204-4563-a6b6-bf50a54f415a None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:09.554384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 106/209] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:09.559834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:09.560302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:09.560471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:09.578243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1c515-9436-44d5-9868-d3f12f06739b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:09.578553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1c515-9436-44d5-9868-d3f12f06739b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:09.578788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1c515-9436-44d5-9868-d3f12f06739b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:09.579881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:09.581024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 107/210] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:09.581196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:09.586508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:09.586791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:09.587110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:09.607140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:09.608378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:09.630334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:09.672466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 104/211] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:09.679254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 108/212] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:09.944473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:09.945117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.043562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61023-980f-48b2-89c9-d6592fe03027 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.056707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7ebc-5a2b-4efe-a2cd-71dda75e212d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.077609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.078259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.078535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.078803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.080029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:10.081526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:10.086440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:10.087677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 109/213] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:10.088991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.096412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.097064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.097349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.097628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.109214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:10.114432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:10.116016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:10.117151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:10.118194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:10.118267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 105/214] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:10.125387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.213717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11343f7-5e08-4eb5-8ff8-f8ca73f9d980 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.246755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea3844-6046-422a-b604-bda5d36e7e02 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.255356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.256474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.256945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.257424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.258385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:10.260174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:10.286333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.286956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.287180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.287405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 110/215] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:10 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:42:10.287517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.288082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:10.288550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:10.289802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab43b8-25b0-427a-9db6-75ac3baac03c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:10.300154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 106/216] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:10.307177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.404039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eac54-03e1-4400-bf73-2006ce702209 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.404724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eac54-03e1-4400-bf73-2006ce702209 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.405195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eac54-03e1-4400-bf73-2006ce702209 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.405296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:10.407811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 107/217] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:10.418203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0554-056f-4da7-bd0d-42bbc27baa89 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.418849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0554-056f-4da7-bd0d-42bbc27baa89 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.419599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0554-056f-4da7-bd0d-42bbc27baa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.435710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84e04-b2a8-41d2-98fd-d366f4c73693 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.445842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-bc7b0554-056f-4da7-bd0d-42bbc27baa89 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:10.447028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bc7b0554-056f-4da7-bd0d-42bbc27baa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:10.485976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.486568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.486921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.487223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] PATH_INFO: `/v3/users/21703cefe4ac484993e5834fb3ba4f7a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.502043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] RBAC: Authorizing `identity:get_user(user_id=21703cefe4ac484993e5834fb3ba4f7a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:10.504079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:10.506723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 111/218] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:10 2022] GET /identity/v3/users/21703cefe4ac484993e5834fb3ba4f7a => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:10.510489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:10.513588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.547037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 108/219] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:10.632647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec96d8-b3d1-4a92-ab20-e0ed51ba15a7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.667036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168bc1c-4329-40aa-9afd-20238c799127 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.667549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168bc1c-4329-40aa-9afd-20238c799127 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.667549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168bc1c-4329-40aa-9afd-20238c799127 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.669179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 109/220] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:10.674865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.674865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.675120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.675979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:10.676613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:10.676889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:10.677157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:10.687929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:10.699749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:10.701018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:10.759901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:10.761302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:10.797995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 110/221] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:10.831982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.942018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 112/222] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:10.947673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:10.948660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb06468-7304-4b1e-800c-409758dbc61b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:10.967493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.017702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.018243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.018529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.018802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.021686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.038904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:11.043812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:11.044696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:11.045471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:11.048439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 111/223] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:11.051931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.130027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1647df-b0fe-4ed7-a944-3cc19b75d232 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.188382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.189803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.190298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.190755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.191397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a119-a343-4257-88b6-3204ad050c39 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.192354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.195204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.197446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 113/224] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:10 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:11.198493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.240185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.241356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.241801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.242263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.243361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:11.244753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.246327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.257883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 112/225] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:11.259291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.301359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.314728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db2d-6599-4bd8-9ded-bca6f391cc3d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.324189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.355672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.356412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.356752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.357088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.368695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:11.374071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:11.374933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:11.375750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:11.376474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.399512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aed5b-cb20-401b-abff-e22457ce78fc None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.440114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.440858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.441224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.441598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.442517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=21703cefe4ac484993e5834fb3ba4f7a) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:11.442861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.444442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.450479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 113/227] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/role_assignments?user.id=21703cefe4ac484993e5834fb3ba4f7a&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:11.451445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.511783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b336-9eb5-43e4-b4bb-968de7bb3f57 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.557165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.558338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.558751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.559191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.560510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.560921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.563389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.565614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 115/228] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:11.568894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b58b0-1aaf-4d48-b02a-87d411b8345f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.572626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.608480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.609263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.609648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.609983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.610853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:11.611439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.612945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.621273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:11.624046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 114/229] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:11.631636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.739888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329cae7-070c-473e-97ac-ca2855e2c266 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.804854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.805523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.805807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.806083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.806624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa0a1-3ced-4cad-8183-c47aa7b9fd3e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:11.811834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.813333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.815718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 116/230] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:11.821551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.877431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:11.878082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:11.878363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:11.878639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:11.880094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:11.881590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:11.887428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 115/231] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:11.894754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:11.944964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:12.008459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d2f1-f77b-4121-8984-212cc396bdcc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:12.011301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:12.083145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:12.083725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.084002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.084337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/0dccc6c01cb14cde8a0a7f8ed68c7297/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.086831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, user_id=0dccc6c01cb14cde8a0a7f8ed68c7297, role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:12.087275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788cb7-0956-47d7-8c25-90f6b71d6600 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:12.088333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:12.110443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 117/232] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:11 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/0dccc6c01cb14cde8a0a7f8ed68c7297/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:42:12.139453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:12.140111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.140402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.140679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.142003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:12.143816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d5920-6c81-49a3-b3de-01d08f61e362 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:12.145616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 116/233] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:11 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:12.152030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:12.296974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9066b1-f1fa-4ddd-85af-01531e5fbda3 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:12.336378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:12.336974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.337246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.337514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/0dccc6c01cb14cde8a0a7f8ed68c7297/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.339450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, user_id=0dccc6c01cb14cde8a0a7f8ed68c7297, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:12.340964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37b806-9114-43d6-9e7c-1ca48506b45e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:12.365370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 118/234] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:12 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/0dccc6c01cb14cde8a0a7f8ed68c7297/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:12.794455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c620f3-6431-4a85-9fb3-cbb83804a6b3 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.794737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c620f3-6431-4a85-9fb3-cbb83804a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.795032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c620f3-6431-4a85-9fb3-cbb83804a6b3 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.796676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 117/235] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:12.802041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.802505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.802685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.804132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d3ba6-a322-4f6b-9a3f-de26ad2d9831 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.804409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d3ba6-a322-4f6b-9a3f-de26ad2d9831 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.804683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d3ba6-a322-4f6b-9a3f-de26ad2d9831 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.806110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 118/236] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:12.815292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:12.815865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:12.815865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:12.825374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:12.826564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:12.846895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:12.847820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:12.848275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:12.939200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 119/237] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:13.007770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 119/238] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:13.389394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:13.413222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:13.472409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:13.509807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e09a0-f4eb-4569-a159-05854eb267a0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:13.554104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:13.555624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.556299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.556953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.559029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:13.561767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:13.564753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 120/239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:13.570989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:13.599039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:13.677987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa273aba-cea2-4339-945d-5f8e6c1d0f00 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:13.691367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fa6f6-bf40-4bc2-89bb-55c6af29e930 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:13.730146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:13.731274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.731753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.732221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.733127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:13.735250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a745278-f870-4a7f-8b16-df149a560f17 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:13.749372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:13.750011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.750363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.750563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.763228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:13.764294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 121/240] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:13 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:13.765812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a738-7c2c-4c70-ad85-f7c7a35139bc None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.766286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a738-7c2c-4c70-ad85-f7c7a35139bc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.766828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a738-7c2c-4c70-ad85-f7c7a35139bc None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7ae0e45d-d9b3-4c6e-9762-9cbbb384524a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:13.768693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:13.769571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:13.770384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 120/242] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:13.771575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 122/242] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:13.775477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2a738-7c2c-4c70-ad85-f7c7a35139bc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:13.775932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.776226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.776505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.797243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:13.798668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:13.885356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:13.917267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2a738-7c2c-4c70-ad85-f7c7a35139bc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:13.961026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:13.962047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:13.962576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:13.963059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:13.963937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:13.964701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:13.966810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:13.980813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 123/243] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:13.982631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 121/244] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:13.987980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.093926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5cf1f-3ad9-4b43-82aa-09363b2963b1 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.131186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.132194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.132682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.133147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] PATH_INFO: `/v3/users/4755794c1e6a40908b8f6ad7b66af89b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.141750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.150675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] RBAC: Authorizing `identity:get_user(user_id=4755794c1e6a40908b8f6ad7b66af89b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:14.152457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:14.154663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 124/245] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:13 2022] GET /identity/v3/users/4755794c1e6a40908b8f6ad7b66af89b => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:14.160735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.185830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.281273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.288360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766d0d-7a81-4f66-a72e-227765912144 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.326313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.327424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.327871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.328142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.409057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:14.410921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:14.430261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.436158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ea04-bbe1-434d-9fed-882851555541 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.481670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.482303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.482592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.482865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.501430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:14.506926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:14.507839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:14.508622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 125/246] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:14.521792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.553432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 122/247] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:14.558679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.603173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.659147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e172f5-c5c1-4630-85ae-2159387b0de0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.687190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e2156-355c-4e63-8d59-926705659f1c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.703040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.704113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.705017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.705299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.705873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.706564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:14.706868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:14.708366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a67454-7082-4821-8f3d-04915fe8f017 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:14.720745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 126/248] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:14.722265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815f5e-aa7d-4e75-9d2a-b7fdf49a9998 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.722646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815f5e-aa7d-4e75-9d2a-b7fdf49a9998 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.722919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815f5e-aa7d-4e75-9d2a-b7fdf49a9998 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.723678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.724195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 127/249] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:14.724554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.728180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815f5e-aa7d-4e75-9d2a-b7fdf49a9998 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.729097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.729686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.730906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:14.734438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af72f5-b647-4a38-a8bf-c27b9023d887 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:14.736823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 123/250] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:14.739402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c483f-df78-468c-b178-9fe7cfd7c09e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.739894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c483f-df78-468c-b178-9fe7cfd7c09e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.740362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c483f-df78-468c-b178-9fe7cfd7c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.768083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-234c483f-df78-468c-b178-9fe7cfd7c09e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:14.769438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-234c483f-df78-468c-b178-9fe7cfd7c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:14.843664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815f5e-aa7d-4e75-9d2a-b7fdf49a9998 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.860787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 124/251] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:14.862097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916667-4eb7-4739-b68e-bfb82ed29af1 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.862396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916667-4eb7-4739-b68e-bfb82ed29af1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.862827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916667-4eb7-4739-b68e-bfb82ed29af1 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.863870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:14.868635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 125/252] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:14.868635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e916667-4eb7-4739-b68e-bfb82ed29af1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:14.888954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:14.890358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.890896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.891663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] PATH_INFO: `/v3/users/21703cefe4ac484993e5834fb3ba4f7a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.893126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] RBAC: Authorizing `identity:get_user(user_id=21703cefe4ac484993e5834fb3ba4f7a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:14.894869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed2b0-31dc-4c66-b705-e6ba5224313a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:14.896918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 128/253] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/users/21703cefe4ac484993e5834fb3ba4f7a => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:14.902936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:14.903537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:14.904064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:14.924144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:14.925650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:14.989970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e916667-4eb7-4739-b68e-bfb82ed29af1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:14.997038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 129/254] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:14.997953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.006424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:15.034494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.035185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.035509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.035778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.036505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=4755794c1e6a40908b8f6ad7b66af89b) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:15.037024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:15.038371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:15.044832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 126/255] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/role_assignments?user.id=4755794c1e6a40908b8f6ad7b66af89b&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:15.130803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3ee92-caab-4b8c-8068-a79cd99b22aa None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:15.152587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:15.169120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.169775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.170051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.170318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.171607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:15.173549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:15.175441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:14 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:15.181483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.280286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.301820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa3592-1675-4cb2-a44b-3ee962daad97 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:15.342941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.343662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.343940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.344216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/21703cefe4ac484993e5834fb3ba4f7a/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.347124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, user_id=21703cefe4ac484993e5834fb3ba4f7a, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:15.349123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:15.372817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 127/257] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:15 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/21703cefe4ac484993e5834fb3ba4f7a/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:15.446940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.500057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea728-b5f2-460c-b4ef-7185f79cc3e9 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:15.548427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:15.570521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.571217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.571653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.571924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.585378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:15.592754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:15.593828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:15.594979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 131/258] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:15.601345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.672257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:15.681152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c16e46-bb2c-43df-b81e-e91639b4eec5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:15.741108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff7a48-775e-4ca6-8c9b-dd94de3ebf45 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:15.764692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.765614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.766036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.766610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.791950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:15.797826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:15.799570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:15.800692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:15.811229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 128/259] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:15.813511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.828726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:15.829744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:15.830144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:15.830536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:15.831382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:15.831838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:15.833394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:15.846782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 132/260] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:15.852011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:15.966902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:15.976411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47557885-4519-4fd9-903d-69444e6f4530 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.025048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.025858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.026079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.026349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.027104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:16.027471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.028939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.039420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 129/261] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:16.044605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.132266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce531-5f67-4f7e-a87d-765a4d24f4b0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.146894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:16.204105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.204745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.205138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.205408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.207537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.209024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.210856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 133/262] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:15 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:16.232912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.243584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13878071-b03d-4d58-9d47-d95846a9fd1b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.312916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.313831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.314290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.314569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] PATH_INFO: `/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.315894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] RBAC: Authorizing `identity:get_user(user_id=0dccc6c01cb14cde8a0a7f8ed68c7297)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.317338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.321086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 130/263] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/users/0dccc6c01cb14cde8a0a7f8ed68c7297 => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:16.326630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.365827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:16.436383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b57a0-917a-4356-a0e9-9a39d4105c8a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.450378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eba95-fc4e-4720-bd5a-aa88b242aa81 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.488421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.488950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.490013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.490286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.502107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.502761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.503090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.503330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.575183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.577011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.579136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:16.605773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:16.629154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.631093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.682599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 131/264] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:16.690481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.803274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bd74-2856-47db-95ba-83942016e7f6 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:16.835055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:16.843391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 134/265] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:16.849234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:16.850050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:16.850420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:16.850783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:16.853311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:16.854866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:16.856907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 132/266] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:16.858472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.863451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:16.980621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aad78-280d-4df1-9d41-ff3862c38bc2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.021468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:17.022249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.022539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.022812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.023588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=0dccc6c01cb14cde8a0a7f8ed68c7297) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:17.023990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:17.025598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69016a2-1b79-467e-8e55-630f5ea87537 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:17.032466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 133/267] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/role_assignments?user.id=0dccc6c01cb14cde8a0a7f8ed68c7297&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:17.038536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:17.083235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca35a0-09a2-4a90-a5d2-6b4cde2d1236 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.130938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:17.131777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.132057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.132328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.133589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:17.135113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:17.137198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 135/268] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:16 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:17.140653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1116e-505e-4dfe-9949-c247a9adff87 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.140936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1116e-505e-4dfe-9949-c247a9adff87 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.141212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1116e-505e-4dfe-9949-c247a9adff87 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.142845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 134/269] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:17.146621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:17.166894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.167222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.167514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.200179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:17.201745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:17.262859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b04e-5f61-43ac-a6e4-1eb5a1417d78 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.303490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:17.304102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.304380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.304640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.305681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=0dccc6c01cb14cde8a0a7f8ed68c7297) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:17.305988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:17.308526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede900d-c22c-4506-b55b-d557dc90c73b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:17.316006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 136/270] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:17 2022] GET /identity/v3/role_assignments?user.id=0dccc6c01cb14cde8a0a7f8ed68c7297&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:17.331571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851242c7-f1f7-40ad-a7e7-004260194da0 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.331983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851242c7-f1f7-40ad-a7e7-004260194da0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.332367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851242c7-f1f7-40ad-a7e7-004260194da0 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.335421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 137/271] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:17.343199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.343540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.343813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.347178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:17.361525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:17.376224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:17.376617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 135/272] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:17.377553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:17.462526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 138/273] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:17.703366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:17.817545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6bb94-eb88-49a7-a4ca-ee08f0787a43 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.826681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:17.857035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:17.857670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.857946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.858228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.859348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:17.860842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:17.862578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 136/274] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:17.867972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:17.933898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:17.948973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127ca9-e4cc-4d8e-8f2a-88d5bf324c85 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.982026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f02dc-3be8-4276-bca3-ff6e68c7179e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:17.989178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:17.989806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:17.990081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:17.990345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:17.993156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.002181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-68c484ee-5d15-4af2-98a1-20098ca4529e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:18.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:18.008062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:18.008924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 139/275] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:18.009628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256e85-327b-40c9-acb6-02d73b7ab67d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.009899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256e85-327b-40c9-acb6-02d73b7ab67d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.010162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256e85-327b-40c9-acb6-02d73b7ab67d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.012128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 140/276] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:18.015937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256e85-327b-40c9-acb6-02d73b7ab67d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.025025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.025650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.026177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.026177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.026832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:18.028323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f5b47-197e-4851-a1fd-75cf810ebd93 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:18.060454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 137/277] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:17 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:42:18.060454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.060454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.060454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.075118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:18.076559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:18.152473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256e85-327b-40c9-acb6-02d73b7ab67d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.171673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 138/278] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:18.194729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.195577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.195955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.196382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.197181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:18.197665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:18.199282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffa9b3-151e-4830-921c-4954d18cf358 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:18.209902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 141/279] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:18.215649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.262765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.313893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9370-f045-4e9e-ac77-4e7cfa1fee5a None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.314259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9370-f045-4e9e-ac77-4e7cfa1fee5a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.314599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9370-f045-4e9e-ac77-4e7cfa1fee5a None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.316186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 142/280] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:18.321551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.321922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.322266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.325045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713672f-290a-4d95-a25b-3aade3fb6b99 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.347424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:18.348633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:18.375432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.376131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.376567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.376899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] PATH_INFO: `/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.396856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea24e9fdd9e4b88b0b83ca6bb1d5314)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:18.398799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:18.401575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 139/281] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:18.407810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.426321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.457533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 143/282] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:18.464254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.527720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.531194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e562c1-444d-4664-ba04-dad067978ce7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.569017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.569912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.570284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.570763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.571595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, user.id=4ea24e9fdd9e4b88b0b83ca6bb1d5314, scope.system=all) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:18.571905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:18.573358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:18.581972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 140/283] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/role_assignments?user.id=4ea24e9fdd9e4b88b0b83ca6bb1d5314&scope.system=all&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:18.599065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d5f9-1444-44bf-b2e4-5c56158989c1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.637316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.637966 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.638244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.638527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.650125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:18.655101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:18.656007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:18.656834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 144/284] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:18.664145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.784150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.825959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a4a74-685b-49aa-a614-1a72cfd24301 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.856638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.866364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.866980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.867442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.867881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:18.868693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:18.869131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:18.870642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:18.881566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 141/285] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:18.888504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:18.932790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:18.935598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d4b6-988a-41ef-985c-57f1162a03b8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:18.994791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:18.995692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:18.996136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:18.996706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.008412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:19.013904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:19.014989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:19.016008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:19.018182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 145/286] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:19.023859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.142911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c82dc-0fd4-4ac2-8cb4-97878e049dfa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.145199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7964-a5d3-49c8-bfe9-9c4af5f53383 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.155042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:19.185473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.186365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.186782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.187220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.188054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:19.188776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.190615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.201804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 146/287] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:19.212141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.214463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.215133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.215439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.217197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] PATH_INFO: `/v3/users/4755794c1e6a40908b8f6ad7b66af89b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.217197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] RBAC: Authorizing `identity:get_user(user_id=4755794c1e6a40908b8f6ad7b66af89b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.218409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.221985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 142/288] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:18 2022] GET /identity/v3/users/4755794c1e6a40908b8f6ad7b66af89b => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:19.234256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.372347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d82105-725b-43b4-825d-f16ce672d055 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.378950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:19.386774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d7dc4-5a19-4d2b-8835-3cbf89aa69ff None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.411374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.411956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.412204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.412499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] PATH_INFO: `/v3/users/21703cefe4ac484993e5834fb3ba4f7a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.413761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] RBAC: Authorizing `identity:get_user(user_id=21703cefe4ac484993e5834fb3ba4f7a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.415275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.417159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 147/289] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:19 2022] GET /identity/v3/users/21703cefe4ac484993e5834fb3ba4f7a => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:19.422969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.425805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.426703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.427242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.427712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.429032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.430682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.437319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 143/290] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:19 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:19.438756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.468008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:19.576393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370b94-ef90-4706-a5c6-87ee2c321a5e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.618870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4e463-be6f-4843-8d50-3cc915cbda1c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:19.658178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.659282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.659755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.660546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/4755794c1e6a40908b8f6ad7b66af89b/roles/2ef85303f3c64cd59d98007e7782d03b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.662909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, user_id=4755794c1e6a40908b8f6ad7b66af89b, role_id=2ef85303f3c64cd59d98007e7782d03b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.664589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.681372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 144/291] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:19 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/users/4755794c1e6a40908b8f6ad7b66af89b/roles/2ef85303f3c64cd59d98007e7782d03b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:42:19.687030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:19.687030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:19.687030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:19.687030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:19.799506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:19.801117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:19.844269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:19.910686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 148/292] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:19.916362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:19.947942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:20.039595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537335d0-d205-4b22-82ce-35676a79585f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:20.074645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:20.075606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.076080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.076520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.077835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:20.079626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80013af4-63b3-44f6-a36a-6aac1a984927 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:20.081730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 145/293] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:19 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:20.087856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:20.194940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cbf49-df0a-4293-a36c-ad1e6fb783ad None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:20.234449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:20.235172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.235450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.235718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.236458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=21703cefe4ac484993e5834fb3ba4f7a) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:20.236760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:20.238222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d29e-4644-44c9-8182-aa8d6c75f842 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:20.244800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 149/294] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:20 2022] GET /identity/v3/role_assignments?user.id=21703cefe4ac484993e5834fb3ba4f7a&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:20.672616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11d97-d7b2-4e0a-bea5-c72c1cea3ed1 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.673167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11d97-d7b2-4e0a-bea5-c72c1cea3ed1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.673167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11d97-d7b2-4e0a-bea5-c72c1cea3ed1 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.674738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 146/295] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:20.680178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e33de-324d-4183-a1cc-eba4bb6a13d2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.680459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e33de-324d-4183-a1cc-eba4bb6a13d2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.680726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e33de-324d-4183-a1cc-eba4bb6a13d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.687839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:20.703272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e94e33de-324d-4183-a1cc-eba4bb6a13d2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:20.704543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e94e33de-324d-4183-a1cc-eba4bb6a13d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:20.728827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db8731-2ee0-4e5a-9713-24266b5852b9 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.729446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db8731-2ee0-4e5a-9713-24266b5852b9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.729946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db8731-2ee0-4e5a-9713-24266b5852b9 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.732749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 147/296] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:20.737472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:20.737759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:20.738103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:20.760182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:20.761387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:20.793738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 150/297] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:20.842175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 148/298] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:21.052669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07ef4-ce19-4463-aab2-f6fd23c38a74 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.053145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07ef4-ce19-4463-aab2-f6fd23c38a74 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.053523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07ef4-ce19-4463-aab2-f6fd23c38a74 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.055270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 151/299] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:21.059141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.061438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.061712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.061977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.081470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:21.082679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:21.083861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.165396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 152/300] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:21.166076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.179680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a8e0-41b9-4b9a-bb00-bf3bd1142c5f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.226553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.228033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.228516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.228960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.229341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.243807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:21.250217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:21.251381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:21.252196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:21.254161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 149/301] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:21.262884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.277779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210aed0-94e0-471c-871d-dd300d1b064a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.315390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.316299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.316672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.317030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.318213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.319968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.321642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 153/302] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:21.327191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.358656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.385677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a839-5f5f-4d1d-99e7-30538f9c425e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.436183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.436883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.437228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.437565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.438461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] RBAC: Adding query filter params (name=reader) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:21.438958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.440445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.445582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29945-21bb-4f48-bec9-133e99e2cf02 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.449540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 150/303] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:21.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.476100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.482484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.483371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.483817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.484275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] PATH_INFO: `/v3/groups` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.485165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.486750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.515035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 154/304] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Aug 30 00:42:21 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:42:21.515035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.589122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c077efe-aca0-4791-9d7a-7a77ad7be89f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.644397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.645052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.645330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.645600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] PATH_INFO: `/v3/users/815c0682d9b1405582a3b6be94b860b5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.663104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.668473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a061f-a84f-4e28-9bf8-2d0e0e5d711d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.671111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] RBAC: Authorizing `identity:get_user(user_id=815c0682d9b1405582a3b6be94b860b5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.673074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.675924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 151/305] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/users/815c0682d9b1405582a3b6be94b860b5 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:21.683124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.709606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.710550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.711088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.711573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.726195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:21.732373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:21.733270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:21.734488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 155/306] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:21.741544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.796273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:21.808304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3de32-7aee-4f63-ba46-e1d646bf128c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.857856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.858795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.859065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.859348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.860164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, user.id=815c0682d9b1405582a3b6be94b860b5, scope.system=all) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:21.860474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.861897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d9eac-6a29-4333-a4a7-294f0fead7c0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:21.862112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.869045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 152/307] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/role_assignments?user.id=815c0682d9b1405582a3b6be94b860b5&scope.system=all&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:21.907865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:21.908802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.909245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.909652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.910514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:21.910953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:21.912622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a6e14-aa38-4327-b7d8-f661a80ac56c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:21.923097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 156/308] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:21.929138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:21.931061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fac9c-7132-4227-a222-7e13bd9d9a62 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.931340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fac9c-7132-4227-a222-7e13bd9d9a62 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.931610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fac9c-7132-4227-a222-7e13bd9d9a62 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.933013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 157/309] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:21.939440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:21.939774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:21.939985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:21.960850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:21.962190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:22.001777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:22.053715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d937-483f-4607-bb39-0b84fe03b663 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:22.056784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 158/310] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:22.097015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:22.097660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:22.097941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:22.098211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] PATH_INFO: `/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:22.099382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea24e9fdd9e4b88b0b83ca6bb1d5314)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:22.100883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:22.102775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 153/311] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:21 2022] GET /identity/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:22.108702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:22.154392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:22.215892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9210085-bd1f-48cf-9131-e6cc06e6ef29 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:22.250135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:22.250871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:22.251204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:22.251428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] PATH_INFO: `/v3/system/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:22.252848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ea24e9fdd9e4b88b0b83ca6bb1d5314, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:22.254248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:22.264646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 159/312] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Aug 30 00:42:22 2022] PUT /identity/v3/system/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:22.348676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:22.467344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b74f-00aa-4ba4-894d-24b779080334 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:22.502137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:22.506491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:22.507176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:22.507455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:22.507726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:22.518935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:22.524400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:22.525260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:22.526066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 154/313] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:22.532881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:22.641585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a220-478b-417c-8758-3a5d161cd9c6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:22.649088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:22.682135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:22.682679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:22.682952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:22.683258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:22.683948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:22.684241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:22.685661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:22.695551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 160/314] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:22.703270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:22.807166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36cf9a-a95c-40ec-a169-84e137d747ea None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:22.843329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:22.843974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:22.844353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:22.844767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] PATH_INFO: `/v3/users/4755794c1e6a40908b8f6ad7b66af89b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:22.846374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] RBAC: Authorizing `identity:get_user(user_id=4755794c1e6a40908b8f6ad7b66af89b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:22.847858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:22.849816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 155/315] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:22 2022] GET /identity/v3/users/4755794c1e6a40908b8f6ad7b66af89b => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:22.856403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:22.963743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326480d3-71d6-4429-bd2a-794922da0b55 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:23.001687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:23.002812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.003326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.003785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.040602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:23.079354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:23.081257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:23.185904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 161/316] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:23.192192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:23.254465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:23.301442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e1ced-efc7-433e-b41a-9872f0989d62 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:23.338679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:23.339350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.339633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.339955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.341099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:23.342615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611f19-3a53-41b1-bef2-14a518804f46 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:23.345108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 156/317] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:23 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:23.351779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:23.426054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:23.462998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e83cb-9529-48f8-871a-e307a666c5eb None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:23.499592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:23.500196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.500728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.500728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.501656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6, user.id=4755794c1e6a40908b8f6ad7b66af89b) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:23.501956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:23.503616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13d21-9477-459b-bb74-d7f70da892ff None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:23.510173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 162/318] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:23 2022] GET /identity/v3/role_assignments?user.id=4755794c1e6a40908b8f6ad7b66af89b&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:23.578948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f428-2267-4ec0-b597-53e323656df8 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.580129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f428-2267-4ec0-b597-53e323656df8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.580609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27f428-2267-4ec0-b597-53e323656df8 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.582566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 157/319] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:23.588665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.588939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.589213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.610020 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:23.611429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:23.688974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 163/320] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:23.954211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:23.954415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93418cd-10e4-440c-a9cb-f22091a54979 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.954827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93418cd-10e4-440c-a9cb-f22091a54979 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.955151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93418cd-10e4-440c-a9cb-f22091a54979 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.956361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 158/321] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:23.961430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:23.961716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:23.961992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:23.981213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:23.982445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:24.025124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.071075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 159/322] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:24.082370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fefc1-ed72-4d2c-bc53-c98db93bd38c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.123238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.124216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.124685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.125136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.126570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:24.128270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded382c2-d1f9-428d-a0e8-bd95e1ca6bb6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:24.130034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 164/323] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:24.135853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.197504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.241224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232213a-91cb-49d1-bff4-3b81bad0db58 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.243525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170bab2-067c-40ad-883b-c2ce9918efb8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.243817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170bab2-067c-40ad-883b-c2ce9918efb8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.244092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170bab2-067c-40ad-883b-c2ce9918efb8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.245574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 165/324] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:24.250335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.250611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.250885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.269408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:24.270662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:24.278551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.279195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.279471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.279737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] PATH_INFO: `/v3/groups` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.280417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:24.282092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:24.307043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 160/325] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Aug 30 00:42:24 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:42:24.348597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.357628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 166/326] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:24.477861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78d19-dba9-4daa-917d-349ceb425226 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.498897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.521339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.521984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.522295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.522544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.533134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:24.538389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:24.539731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:24.540747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 161/327] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:24.545411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.618187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.632954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.666246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82752535-1659-456d-a6a8-2aba2c5843be None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.706315 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.706930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.707278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.707525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.708207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:24.708507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:24.709912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:24.718777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 167/328] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:24.725637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.773592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee43e2-74ca-4f9b-b4b7-862d31068b42 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.817777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.818422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.818704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.818979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.835614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:24.841273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:24.842130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:24.842925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:24.844091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 162/329] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:24.852897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:24.875229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb3db3-f4d2-4959-b7e0-803662ad94b0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:24.908804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.919756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:24.929210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:24.929858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:24.930133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:24.930398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] PATH_INFO: `/v3/users/815c0682d9b1405582a3b6be94b860b5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:24.931734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] RBAC: Authorizing `identity:get_user(user_id=815c0682d9b1405582a3b6be94b860b5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:24.933178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:24.935024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 168/330] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:24 2022] GET /identity/v3/users/815c0682d9b1405582a3b6be94b860b5 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:24.948202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:25.028241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79c448-b6fa-486d-997e-a0d40c165c98 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:25.079557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc32c3-9064-4e4b-8463-803a5b15b93f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:25.082098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:25.082735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:25.083195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:25.083469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:25.084155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:25.084459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:25.085872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:25.098468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 163/331] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:25.105623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:25.131313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:25.131988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:25.132289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:25.132545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] PATH_INFO: `/v3/system/users/815c0682d9b1405582a3b6be94b860b5/roles/2ef85303f3c64cd59d98007e7782d03b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:25.134240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=815c0682d9b1405582a3b6be94b860b5, role_id=2ef85303f3c64cd59d98007e7782d03b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:25.135675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:25.145250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 169/332] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Aug 30 00:42:24 2022] PUT /identity/v3/system/users/815c0682d9b1405582a3b6be94b860b5/roles/2ef85303f3c64cd59d98007e7782d03b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:25.234620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:25.238125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c07b-06c3-415f-b803-8168cc2f9ea3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:25.277130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:25.277775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:25.278051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:25.278427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] PATH_INFO: `/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:25.279620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] RBAC: Authorizing `identity:get_user(user_id=4ea24e9fdd9e4b88b0b83ca6bb1d5314)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:25.281095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b58f0-6576-4c86-9b99-34c1d02aa4da None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:25.283475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 164/333] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:25 2022] GET /identity/v3/users/4ea24e9fdd9e4b88b0b83ca6bb1d5314 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:25.289392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:25.362981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:25.413574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95e09a-ebbb-456f-bd2b-bb3d275d91af None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:25.451164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:25.451790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:25.452071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:25.452342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:25.453082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] RBAC: Adding query filter params (role.id=fd7cb5f266b5465eb35b727b0d541b07, user.id=4ea24e9fdd9e4b88b0b83ca6bb1d5314, scope.system=all) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:25.453423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:25.454893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45bec4-7c3d-4a37-b1fe-e6567653e6c8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:25.467591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 170/334] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 30 00:42:25 2022] GET /identity/v3/role_assignments?user.id=4ea24e9fdd9e4b88b0b83ca6bb1d5314&scope.system=all&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:26.949959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d07e-7895-4f95-a3e1-76d1443ae0c0 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:26.951078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d07e-7895-4f95-a3e1-76d1443ae0c0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:26.951560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d07e-7895-4f95-a3e1-76d1443ae0c0 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:26.953114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 165/335] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:26.964835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:26.965769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:26.966239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:26.990532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:26.992118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:27.068818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041b91-cfe4-42fa-8b0b-37437205e327 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.069141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041b91-cfe4-42fa-8b0b-37437205e327 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.069371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041b91-cfe4-42fa-8b0b-37437205e327 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.070859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 166/336] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:27.076756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.077037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.077315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.080138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 171/337] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:27.100375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:27.101603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:27.181264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 167/338] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:27.458858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:27.514564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:27.586141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20eb8-6c9c-45cd-8158-c7e2f0e0f2c0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:27.623834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:27.629829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:27.630727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.631265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.631747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.652943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:27.660874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:27.661788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:27.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 172/339] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:27.671642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:27.758723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:27.766912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f382b9b-78f8-4de5-9088-a09625fa85ad None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:27.847053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:27.848215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.850168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.851759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.879395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a63d5eb9-f93c-4f42-8a9c-8e35e0975d31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:27.893392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:27.894352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:27.895163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:27.897568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 168/340] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:27.901559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccba4c3-3fd7-4aea-a7b2-891f19f47583 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.903283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccba4c3-3fd7-4aea-a7b2-891f19f47583 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.903766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccba4c3-3fd7-4aea-a7b2-891f19f47583 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.906257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 169/341] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:27.908837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccba4c3-3fd7-4aea-a7b2-891f19f47583 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:27.934084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a047f52-ab5a-497c-aea3-3955a6338fdc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:27.937893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:27.992251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:27.992886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:27.993138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:27.993413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:27.994092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:27.994394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:27.995980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa401f75-050b-4386-bc90-b183918ec55c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.011046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 173/342] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:28.011046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc162-4b27-4374-b15c-8ead91b8e80b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.011046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc162-4b27-4374-b15c-8ead91b8e80b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.011046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc162-4b27-4374-b15c-8ead91b8e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.028593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1e7bc162-4b27-4374-b15c-8ead91b8e80b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:28.031629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1e7bc162-4b27-4374-b15c-8ead91b8e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:28.061019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccba4c3-3fd7-4aea-a7b2-891f19f47583 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.102439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.104053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.104729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.105432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.106546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:28.106948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.108564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82114f80-5d60-4b89-8679-497525e9968c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.108897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 174/343] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:28.110765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eb78-34d3-471c-86b2-addc09851c3d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.111244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eb78-34d3-471c-86b2-addc09851c3d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.111744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eb78-34d3-471c-86b2-addc09851c3d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.113285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 175/344] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:28.114247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58eb78-34d3-471c-86b2-addc09851c3d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.119217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 170/345] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:28.120883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.121177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.121455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.141755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:28.143108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:28.160214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.219668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 171/346] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:28.221515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.226750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.236956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58eb78-34d3-471c-86b2-addc09851c3d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.275410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.276505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.277073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.277541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.330395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c21681-8e9e-49cd-95ac-43ee7570794d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.355309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.357281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.369752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.370250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.370530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.370804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.391998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.448018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.449504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.463662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.474077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 176/347] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:28.480255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.560331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 172/348] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:28.568195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.587452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e86caf-2c79-4253-afe9-6b8fb360924f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.631048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.631048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.631048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.631048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.631705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.631705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.632689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 177/349] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:28.639479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.649497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.717764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.756461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faec52a-3493-48ea-9dec-18d73d5dc948 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.757259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47ab9-acff-4af9-901c-84bf7223e4b8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.802522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.803211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.803488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.803759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.804894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.806359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.808916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.808916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.808916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.809187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 173/350] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:28.809262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.809330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.817422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:28.819141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36491c5e-f8a6-4359-8047-48c48e5a85ab None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:28.821193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 178/351] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:28.822543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e1b2-e1e8-47e7-9ac3-b105eb4e3177 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.822888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e1b2-e1e8-47e7-9ac3-b105eb4e3177 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.823287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e1b2-e1e8-47e7-9ac3-b105eb4e3177 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:28.824571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 179/352] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:28.825318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e1b2-e1e8-47e7-9ac3-b105eb4e3177 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:28.945404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697725b-4cba-4d2a-b38f-2c15eb68e70e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.951181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e1b2-e1e8-47e7-9ac3-b105eb4e3177 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:28.965885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.987913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:28.988984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:28.989576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:28.989853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:28.990192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.001171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.001867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.002142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.002569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.008645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:29.014527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:29.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 174/353] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:29.016991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.026379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:29.029698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:29.030579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:29.031836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 180/354] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:29.035486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.151487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d967-2b12-4337-96e2-981bb10bd7c2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.166527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc470c7-dd32-46a6-92a2-55698577b0ce None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.213587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.213587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.213587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.213587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.215164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.216595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a963a-7215-4ab8-a46e-028cbe113f98 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.218336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 175/355] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.218591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.223033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.223033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.223033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.224108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.224658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.225116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.226060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=a462dd813727475a9e0d05c4c7c465bc) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:29.226550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.228231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.240005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 181/356] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.241177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.243035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:29.244254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:29.304239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.306281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.343190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 176/357] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:29.344776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.389089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7a510-9076-407e-b89d-5eb357aa5d2f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.455889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.456533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.456827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.457112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.457808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:29.458115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.463789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.475978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 182/358] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.477282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.482001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22369c3-28bb-4f36-a982-97489ab0e4e8 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.520351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.520566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.521127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.521127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.521638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:29.521943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.523372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89753c-87b7-4006-8bfa-80d4e36037ea None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.532669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 177/359] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.534057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4f5b-4dfa-4668-84bc-27f792620ed3 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.534282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4f5b-4dfa-4668-84bc-27f792620ed3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.534501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4f5b-4dfa-4668-84bc-27f792620ed3 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.535749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 178/360] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:29.537262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce4f5b-4dfa-4668-84bc-27f792620ed3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.584580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.613776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419386b-7e75-4e1c-9299-06f36eaafba0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.629046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.653830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce4f5b-4dfa-4668-84bc-27f792620ed3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.654044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.654503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.654778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.655077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.655968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=a462dd813727475a9e0d05c4c7c465bc) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:29.656448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.657912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.666052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 183/361] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.668470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.700531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.701283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.701668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.702015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.797958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.799637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.805911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5c71-ed0a-419e-a557-fc769b6970c1 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:29.849022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:29.849675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.849961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.850243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] PATH_INFO: `/v3/users/815c0682d9b1405582a3b6be94b860b5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.852853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] RBAC: Authorizing `identity:get_user(user_id=815c0682d9b1405582a3b6be94b860b5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:29.854457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b53c56-a79f-464c-8f7d-301d5640a157 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:29.856506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 184/362] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/users/815c0682d9b1405582a3b6be94b860b5 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:29.858658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:29.858949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:29.859243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:29.866894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.883052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:29.885225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:29.918691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 179/363] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:29.919744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:29.942910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:29.978755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 185/364] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:29.979905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.091056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf158fe-87a5-4aad-a033-f66431c2ad03 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.110727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d838-a6ef-4d1e-8385-29eb9b8509f0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.130057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.130673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.130948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.131264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.131996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] RBAC: Adding query filter params (role.id=2ef85303f3c64cd59d98007e7782d03b, user.id=815c0682d9b1405582a3b6be94b860b5, scope.system=all) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:30.132298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.134082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.148691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 186/365] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/role_assignments?user.id=815c0682d9b1405582a3b6be94b860b5&scope.system=all&role.id=2ef85303f3c64cd59d98007e7782d03b => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:30.149799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.187449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.210522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.211271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.211547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.211822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.240815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.247398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:30.252634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:30.254729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:30.256710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 180/366] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:30.276100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.277447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4b326-f9d1-4c9f-9aa0-e89c198a1250 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.314915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.315648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.315927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.316201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.317389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.318935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.325501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 187/367] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:30.326830 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.444043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2d665-1524-4439-8f3e-32fa358fd66c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.446549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ca34d-44a9-4823-8ad6-52f949da23c7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.447049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.482257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.482951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.483258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.483578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.487164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.488841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.490600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 188/368] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:30.490885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.491910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.492383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.492840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.493953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:30.494468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.496443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.497635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.509287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 181/369] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:30.510151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.588843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.624180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb739f35-34a6-4f8b-90e6-1533bcafd29a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.641248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d8031-46c6-4efb-b19d-1be52391a3fc None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.665692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.666231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.666507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.666776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.667544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:30.667849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.669326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.679795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 189/370] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:30.681025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.682979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.683677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.684188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.684491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.701999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:30.708228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:30.709230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:30.710011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 182/371] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:30.716171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:30.790330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.865650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23978f-fd8a-4188-b10a-9742e945bb2c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.904722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:30.933932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:30.934464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:30.934740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:30.935059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:30.935833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:30.936132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:30.938503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:30.949955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1adc24-db27-46ef-8675-090b3054f738 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:30.950163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 183/372] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:30.964450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.004700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.005479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.005926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.006221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.087689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83d4e-8ec6-4149-8c60-04a936308ef0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.109996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.111685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.125260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.126237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.126662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.127087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.154292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.182357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.204151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.205885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.307580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 190/373] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:31.313932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.314397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 184/374] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:31.317818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.405087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.428308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457070-e320-48d9-8167-72e550d68e80 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.443198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7427-b28a-4129-ae60-d0d74bb54bed None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.466319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.467321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.467810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.468272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] PATH_INFO: `/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.469892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] RBAC: Authorizing `identity:get_project(project_id=aa5e798ada384a4f94fc4eb7a83dd47f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.471873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.474016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 185/375] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:31.479762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.488366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.499711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.500670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.501136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.501670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.504058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.505722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.510557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 191/376] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:31.516740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.682791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4151ca-7537-4187-af2f-dd4f863e6bfd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.708094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a8fc2-12b1-448a-8b1b-cb6a1168f2f8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.752213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.753339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.753825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.754299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.755850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.757533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737f039-8beb-408f-9405-d94cd0ece6d0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.759404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 192/377] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:31.761718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405d28-7172-4ff7-9301-2db411c45fa5 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.762231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405d28-7172-4ff7-9301-2db411c45fa5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.763307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1405d28-7172-4ff7-9301-2db411c45fa5 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.765451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 193/378] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:31.766701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405d28-7172-4ff7-9301-2db411c45fa5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.784246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.784480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.784695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.784906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] PATH_INFO: `/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.791698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] RBAC: Authorizing `identity:get_group(group_id=7cf8df1de7b24f7dacacd3adc0ba8b91)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.793715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a900f-496f-4d5c-87ad-27be09f0bffc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.795439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 186/379] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91 => generated 229 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:31.796753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.797131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.797407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.834269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:31.836929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:31.851734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.887506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:31.917781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1405d28-7172-4ff7-9301-2db411c45fa5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:31.921448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 187/380] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:31.925757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:31.958027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:31.959397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:31.959869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:31.960328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:31.961245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:31.961773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:31.963877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e82c-e6d5-490a-a373-c6b38fdec5c8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:31.971168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 194/381] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:32.111131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a28f7-72d3-45f3-ae75-28f6c731b08e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:32.161347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:32.162002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.162342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.162610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.163373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] RBAC: Adding query filter params (group.id=7cf8df1de7b24f7dacacd3adc0ba8b91, role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=aa5e798ada384a4f94fc4eb7a83dd47f) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:32.163678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:32.165121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb580f7d-731d-4c0a-860b-191df39723d9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:32.175820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 188/382] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:31 2022] GET /identity/v3/role_assignments?group.id=7cf8df1de7b24f7dacacd3adc0ba8b91&scope.project.id=aa5e798ada384a4f94fc4eb7a83dd47f&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:32.243931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748eba-da71-4cae-9e8e-ecc427216aa7 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.243931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748eba-da71-4cae-9e8e-ecc427216aa7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.243931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5748eba-da71-4cae-9e8e-ecc427216aa7 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.244998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 195/383] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:32.250276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.250577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.250819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.272751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:32.274087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:32.276494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:32.367508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5748eba-da71-4cae-9e8e-ecc427216aa7 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:32.369723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 189/384] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:32.533600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5748eba-da71-4cae-9e8e-ecc427216aa7 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:32.568718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:32.575090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:32.575803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.576239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.576536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.591846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:32.598087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:32.599048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:32.599848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 196/385] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:32.608146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:32.767600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653c4f-9f5b-4e44-960a-bbac0b5d5343 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:32.808928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:32.809541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.809817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.810088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.810749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:32.811065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:32.812520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:32.817073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:32.818169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:32.826368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 190/386] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:32.833866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:32.944593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9dcf5-73f3-41f1-ad53-2d545d66ab26 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:32.995736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:32.996393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:32.996666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:32.996932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:32.998098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:32.999686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.001474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 191/387] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:32 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:33.008074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.024554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc08a2-4d25-4dfc-b767-729e93ceee31 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.073986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.074708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.074992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.075289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.093305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:33.103231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:33.104153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:33.104940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:33.105543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:33.107050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 197/388] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:33.112258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.136807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25b6dc-c875-42b3-9b9c-82965db3a652 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.171336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.172164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.172443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.172716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.173886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.175373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.176999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 192/389] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:33 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:33.182746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.216804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:33.317677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3a733-472c-4f90-80b5-5624aabaed01 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.327739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67299f7-f7eb-420b-9993-57910d683179 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.381162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.381788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.382065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.382338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.383823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:33.384129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.385540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2e3da-7b41-4abf-946e-fd4b2893a03d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.393215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.393897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.394179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.394456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.402301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, group_id=cd9de48bc0754f28a3d2fe35304aed1d, role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.402301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.408406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 198/390] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:33.411704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1e07-d46d-4836-b160-afc7a9a1fac5 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.411704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1e07-d46d-4836-b160-afc7a9a1fac5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.411704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f1e07-d46d-4836-b160-afc7a9a1fac5 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.411921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 199/391] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:33.417378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.417648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.417925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.419417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:33.433258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 193/392] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 30 00:42:33 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:42:33.434420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.446796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:33.449440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:33.568246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546e961-5ff8-4747-80f3-d1690c0bf14c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.590439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 200/393] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:33.608526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.609748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.610316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.610831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.612363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.615091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.618858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 194/394] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:33 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:33.626783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.682101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:33.740779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4abc5-3f38-40bb-b5aa-88f0059abce5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.777525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.778449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.778877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.779309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.780690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.782474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.784620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 201/395] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:33 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:33.790655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:33.857869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:33.898026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b5df2-8a28-4cc3-bee3-c79d52be4cb7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:33.934099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:33.935058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:33.935517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:33.935937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:33.938300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] RBAC: Authorizing `identity:create_grant(project_id=a462dd813727475a9e0d05c4c7c465bc, group_id=cd9de48bc0754f28a3d2fe35304aed1d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:33.939981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:33.957078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 195/396] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 30 00:42:33 2022] PUT /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:34.053192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:34.180141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b85f-c0c4-4471-9861-cd522dd3162d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:34.196441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:34.223803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:34.224458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.224735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.225001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.236751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:34.241629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:34.242636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:34.243995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 202/397] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:34.251035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:34.357994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258d18f-2c86-46c1-8c76-f0a4d8acb6af None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:34.382621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:34.399502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:34.401020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.401680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.402019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.402643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:34.402947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:34.404380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:34.423301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 196/398] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:34.426725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:34.571167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf535c-d1a2-4164-8bc4-0445781a8e88 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:34.619271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:34.619917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.620200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.620476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.621646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:34.628486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c90ab-f5d9-427f-a5b6-b74ff40e0ce2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:34.630505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 203/399] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:34 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:34.636221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:34.653069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd6e2-f9cd-44bf-b87e-0f1b06b8b6a6 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.653514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd6e2-f9cd-44bf-b87e-0f1b06b8b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.653940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befd6e2-f9cd-44bf-b87e-0f1b06b8b6a6 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.656452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 204/400] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:34.662593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908453ed-c311-4fea-944f-0e0139a75519 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.663204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908453ed-c311-4fea-944f-0e0139a75519 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.663616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908453ed-c311-4fea-944f-0e0139a75519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.695627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-908453ed-c311-4fea-944f-0e0139a75519 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:34.696854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-908453ed-c311-4fea-944f-0e0139a75519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:34.747342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:34.770446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13128b-0d85-44fd-934b-455186299058 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:34.785424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 205/401] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:34.787125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2c9e-74b6-4ad3-b18a-5d8a56501fd5 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.787514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2c9e-74b6-4ad3-b18a-5d8a56501fd5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.787982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2c9e-74b6-4ad3-b18a-5d8a56501fd5 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.789439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 206/402] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:34.795347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.795735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.796098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.807332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:34.807981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.808262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.808541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.809689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:34.811231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:34.812912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 197/403] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:34 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:34.817432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:34.818722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:34.819085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:34.883998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:34.892656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 207/404] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:34.939048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4e2a9-5a5b-4504-9ada-a2b464bb5702 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:34.974463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:34.975454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:34.975910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:34.976371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:34.978604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, group_id=cd9de48bc0754f28a3d2fe35304aed1d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:34.980341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:34.997950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 198/405] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 30 00:42:34 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:35.182973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.267559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.326895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:35.330767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bbf5a-6afa-4ea4-bb82-b07b26472827 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:35.374773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:35.375679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:35.375982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:35.376229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:35.387807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:35.392745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae578bfc-2824-42a3-981b-677630e75a12 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:35.404800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:35.405745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:35.406700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 208/406] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:35.414218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.432198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:35.469526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:35.470395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:35.470678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:35.470953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:35.496028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:35.503217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:35.504110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:35.504883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:35.505494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 199/407] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:35.512902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.607838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6a0e-f7f8-4219-a2a9-7d8d812b99f6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:35.684997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:35.687343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:35.687649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:35.688149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:35.689517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:35.689828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:35.691275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:35.694690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff725fc-5448-4391-873d-ccb3a5d859fd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:35.715968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 209/408] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:35.729052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.767763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:35.768730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:35.769185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:35.769640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:35.770421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:35.770819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:35.771466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:35.772410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:35.786232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 200/409] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:35.795743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.882105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73138e02-0ee8-4d7f-b016-a06147b38d1f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:35.912835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:35.929008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:35.930376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:35.930653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:35.930927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] PATH_INFO: `/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:35.932115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] RBAC: Authorizing `identity:get_group(group_id=7cf8df1de7b24f7dacacd3adc0ba8b91)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:35.933641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:35.935658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 210/410] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:35.941383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:35.999068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874e19-2681-4a4d-a8fc-bf2f089de3ef None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:36.041600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:36.042240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.042511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.042775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.046772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:36.049353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:36.052523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 201/411] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:36.058113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:36.171367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534678-82a5-447e-9cbc-c6f4419f610e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:36.174460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:36.188990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:36.211777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:36.212418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.212693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.214763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8242-8379-41b7-ba7b-ffe628d4f483 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:36.216264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.217384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:36.218878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:36.222600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 202/412] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:36 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:36.231910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:36.286176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:36.287230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.287516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.287786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] PATH_INFO: `/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.289206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] RBAC: Authorizing `identity:get_project(project_id=aa5e798ada384a4f94fc4eb7a83dd47f)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:36.291383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78fad4-9217-454b-9893-a7ef4ada1551 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:36.293493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 211/413] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:35 2022] GET /identity/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f => generated 344 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:36.294852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb5b-d4a4-4e8b-af3e-b0176c4a3ff5 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.295217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb5b-d4a4-4e8b-af3e-b0176c4a3ff5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.295493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb5b-d4a4-4e8b-af3e-b0176c4a3ff5 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.298952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 212/414] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:36.299281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13bca-4820-4862-b341-7581b82571f9 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.299562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13bca-4820-4862-b341-7581b82571f9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.299827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13bca-4820-4862-b341-7581b82571f9 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.301247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 213/415] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:36.302032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff13bca-4820-4862-b341-7581b82571f9 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:36.370859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109d00-5881-466e-a175-d338a11d86d4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:36.415759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:36.416536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.416661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff13bca-4820-4862-b341-7581b82571f9 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:36.416879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.417221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.420308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8faf18acd794f498fb24448e9eaa6c6, group_id=cd9de48bc0754f28a3d2fe35304aed1d, role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:36.421868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96500e90-44b8-4e9a-93b3-237126777b98 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:36.436148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 203/416] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 30 00:42:36 2022] PUT /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6/groups/cd9de48bc0754f28a3d2fe35304aed1d/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:36.437500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.437785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.438058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.463757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:36.464227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:36.465121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:36.465361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.465843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.466314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] PATH_INFO: `/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f/groups/7cf8df1de7b24f7dacacd3adc0ba8b91/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.469187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa5e798ada384a4f94fc4eb7a83dd47f, group_id=7cf8df1de7b24f7dacacd3adc0ba8b91, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:36.471368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb7ec9-3075-44ab-942e-34ec0e1ffdb0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:36.489414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 214/417] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 30 00:42:36 2022] PUT /identity/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f/groups/7cf8df1de7b24f7dacacd3adc0ba8b91/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:36.491378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:36.491952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:36.492470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:36.517978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:36.519609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:36.520856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:36.552192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:36.626869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 204/418] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:36.657782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 215/419] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:36.908005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:36.948482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.064637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac530556-e8c2-4a6d-ae4e-eca82523c0e1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.088788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54e1be-256c-444c-a6a4-e53ee4084b76 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.128854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.129498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.129778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.130052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.147595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.148249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.148532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.148812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.160374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:37.165161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:37.166150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:37.166944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 205/420] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:37.168079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:37.172209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:37.173318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:37.174179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:37.174967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:37.179448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 216/421] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:37.186046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.217067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.235454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.284121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463180-313d-4f0b-90c8-eda1c46f2cdf None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.342110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.342722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.343068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.343378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.344293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] RBAC: Adding query filter params (name=member) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:37.344717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:37.347426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:37.360657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 206/422] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:37.367238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.404489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d31ba-ecb6-4f4c-9372-74ca6f65c652 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.440903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.441547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.441821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.442086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.442856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:37.443179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:37.444572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e07de-a410-45d8-89b1-1b4e4ff4e888 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:37.453660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 217/423] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:37.459038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94315b-a0e4-45d6-93e5-485eb697721b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.459038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94315b-a0e4-45d6-93e5-485eb697721b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.459038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94315b-a0e4-45d6-93e5-485eb697721b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.459038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 218/424] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:37.459480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94315b-a0e4-45d6-93e5-485eb697721b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.478580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.496939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39aec-a2ee-4313-a4a8-49d2ad2d047b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.541140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.541747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.542022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.542289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.561200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.611300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94315b-a0e4-45d6-93e5-485eb697721b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.614662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:37.616266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40781c1-65f6-4ca0-9821-41bf5ee077d5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:37.657535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:37.658518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.659883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.660326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.705181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.729776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 207/425] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:37.731167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:37.731523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:37.731800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:37.751198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:37.752383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:37.756759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:37.758397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:37.821627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 208/426] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:37.827881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:37.898400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 219/427] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:37.904551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:37.965341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5da38-3efd-4d39-b29a-f390f90d2343 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:37.985577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.012598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.013582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.014166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.014637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.016052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.018396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.020577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 209/428] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.026437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.109052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eb4be-21c1-4d05-9be8-49f7f7d23005 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.179398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.180662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.182212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.182906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] PATH_INFO: `/v3/projects/a462dd813727475a9e0d05c4c7c465bc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.184318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] RBAC: Authorizing `identity:get_project(project_id=a462dd813727475a9e0d05c4c7c465bc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.186724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.189110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 220/429] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:37 2022] GET /identity/v3/projects/a462dd813727475a9e0d05c4c7c465bc => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:38.196113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.232417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2df9b-a5e5-4986-ad46-5e2503ec0408 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.259436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.275415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.276212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.276551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.277033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.278534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.280426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.283381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 210/430] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.284469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.333680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67172d8e-5560-4e6c-b63f-edfb22c08945 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.348545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.381147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.382116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.382562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.383029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.384951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.387065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.389818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 221/431] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.391215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.423392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900290-d1c0-454e-a5fa-5e6e8dd7d7cd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.461230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.462324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.462812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.463388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.477223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:38.484879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:38.486168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:42:38.486953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:38.494269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 211/432] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:38.495458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.507212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff1a2-be48-439e-b6d0-ada9047c1e90 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.515908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.544002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.544605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.544878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.545144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.545943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=a462dd813727475a9e0d05c4c7c465bc) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:38.546163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.548032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.554654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 222/433] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.555299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.617020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fd590-83ee-4d6b-8641-8a743c3e7f65 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.629132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.657862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.658885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.659487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.660042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.661064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=a462dd813727475a9e0d05c4c7c465bc) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:38.661576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.663383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb91493-f5ec-480d-82d2-6e4950d4797a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.666454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785ca9-2cfc-4212-87bc-5698bf066952 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.671306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 212/434] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=a462dd813727475a9e0d05c4c7c465bc&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.696644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536086-53e0-4fd1-bb23-29cef52f37a7 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.697346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536086-53e0-4fd1-bb23-29cef52f37a7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.697840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84536086-53e0-4fd1-bb23-29cef52f37a7 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.699618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 213/435] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:38.705608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39615-8d35-4ef3-abcb-6ecc846dc2ec None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.706235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39615-8d35-4ef3-abcb-6ecc846dc2ec None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.706701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39615-8d35-4ef3-abcb-6ecc846dc2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.746600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c7b39615-8d35-4ef3-abcb-6ecc846dc2ec None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:38.748458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c7b39615-8d35-4ef3-abcb-6ecc846dc2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:38.761127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.764609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.765337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.765801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.766848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] RBAC: Adding query filter params (name=member) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:38.767382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:38.769032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:38.780059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 223/436] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:38.787850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:38.823781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:38.849067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 214/437] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:38.850975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035a124-4dbd-4fab-9121-0a9923de850e None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.850975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035a124-4dbd-4fab-9121-0a9923de850e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.850975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035a124-4dbd-4fab-9121-0a9923de850e None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.854791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 215/438] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:38.859492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.859698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.859975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.880469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:38.881658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:38.914867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa925850-8af2-4032-ada6-64abd5eeac34 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:38.959406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:38.960124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:38.960402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:38.960726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:38.967110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 216/439] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:39.033993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.035406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.067928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.141799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.144272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 224/440] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:39.149965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.186809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.271481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf265cd-c00e-438c-812b-acb09a27513a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.272887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45715f-5045-4871-b91c-da9b4c1d23d1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.307336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.313286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.313471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.313932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.314212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.314483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.314616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.314891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.315304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.316632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.318833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.320219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 225/441] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:39.320788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.327414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:39.331211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:39.333319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 217/442] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:39.336991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.438900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.441091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8276-7efa-4b6c-bd2e-737ba0fb051b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.446337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fc95-7046-4cad-a3bb-db292b30004a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.476614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.477239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.477522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.477788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.480698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.481329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.481603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.481872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.483132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.484755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.486347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 218/443] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:39.487120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.488969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:39.494166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:39.501112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 226/444] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:39.502077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.559662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.590070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c6ef-a4c4-4d0d-bb42-06d1f69ae43c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.602415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa765e1-3562-49bb-8f64-b26355c90d44 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.626571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.627211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.627484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.627755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.628409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:39.628712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.630090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.635675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.636275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.636585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.636855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.637585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=fd7cb5f266b5465eb35b727b0d541b07, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:39.637890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.638342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 219/445] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:39.639174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.639358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.645460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 227/446] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=fd7cb5f266b5465eb35b727b0d541b07 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:39.646293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.672337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.768402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576f37-70ef-441a-ad20-c025d7f04caa None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.769398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264e09-7f77-45dc-881f-487c6e272186 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:39.796776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.808693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.809317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.809602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.809933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.810548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:39.810853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.810953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:39.811540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:39.811816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:39.812795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.816409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:39.822987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 220/447] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:39.828946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:39.897512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:39.899150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:39.944858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:39.959182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9f16-8222-4c63-a272-70d843b81781 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.004528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.006312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.006312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.006312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.026551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 228/448] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:40.032380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.057400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:40.079228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.080648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.133704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fcb4-e608-40e2-84ba-bc0dbd8d38fc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.165414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:40.168215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.168892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.169250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.169533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] PATH_INFO: `/v3/projects/c8faf18acd794f498fb24448e9eaa6c6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.170679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] RBAC: Authorizing `identity:get_project(project_id=c8faf18acd794f498fb24448e9eaa6c6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.172472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.174155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 229/449] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/projects/c8faf18acd794f498fb24448e9eaa6c6 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:40.179898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.181869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 221/450] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:42:40.187650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.280596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:40.290059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349360-d934-4e7a-9585-445d419296a6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.300768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eaa8-1275-4ab8-b276-5eafcea8e0fd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.325025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.325757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.326098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.326368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] PATH_INFO: `/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.327535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] RBAC: Authorizing `identity:get_group(group_id=cd9de48bc0754f28a3d2fe35304aed1d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.329005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.330668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 230/451] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/groups/cd9de48bc0754f28a3d2fe35304aed1d => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:40.336457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.337351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.337992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.338273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.338550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] PATH_INFO: `/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.339721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] RBAC: Authorizing `identity:get_project(project_id=aa5e798ada384a4f94fc4eb7a83dd47f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.341176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.342796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 222/452] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/projects/aa5e798ada384a4f94fc4eb7a83dd47f => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:40.348617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.410279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:40.441124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48934-09cc-40c6-9fa0-d8c4007b9962 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.456985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2103e-2d1b-4b7b-b2e6-090edc5fbc9c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.475191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.475789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.476062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.476323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.477454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] RBAC: Adding query filter params (group.id=cd9de48bc0754f28a3d2fe35304aed1d, role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=c8faf18acd794f498fb24448e9eaa6c6) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:40.477748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.479209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.485334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 231/453] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/role_assignments?group.id=cd9de48bc0754f28a3d2fe35304aed1d&scope.project.id=c8faf18acd794f498fb24448e9eaa6c6&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:40.490745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.491441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.491718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.491991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] PATH_INFO: `/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.493173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] RBAC: Authorizing `identity:get_group(group_id=7cf8df1de7b24f7dacacd3adc0ba8b91)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.494657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe2f07-17b1-430b-b071-f64f9987f810 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.496342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 223/454] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/groups/7cf8df1de7b24f7dacacd3adc0ba8b91 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:40.502808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:40.531368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:40.616522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceef54-0616-4d6c-a343-8bf9f0dd605c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:40.652920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:40.653527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:40.653806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:40.654076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:40.654806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] RBAC: Adding query filter params (group.id=7cf8df1de7b24f7dacacd3adc0ba8b91, role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=aa5e798ada384a4f94fc4eb7a83dd47f) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:40.655139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:40.656618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000be15-a429-490e-8892-82a25e20ed5e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:40.662538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 232/455] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 30 00:42:40 2022] GET /identity/v3/role_assignments?group.id=7cf8df1de7b24f7dacacd3adc0ba8b91&scope.project.id=aa5e798ada384a4f94fc4eb7a83dd47f&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:42.925287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefd7c-dafa-4a16-ad27-0db7607ab4d6 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:42.925842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefd7c-dafa-4a16-ad27-0db7607ab4d6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:42.925842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefd7c-dafa-4a16-ad27-0db7607ab4d6 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:42.927490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 224/456] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:42.932528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6bee-3577-40c0-99c3-f52242743bd4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:42.932944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6bee-3577-40c0-99c3-f52242743bd4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:42.933107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6bee-3577-40c0-99c3-f52242743bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:42.956989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d03b6bee-3577-40c0-99c3-f52242743bd4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:42.958199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d03b6bee-3577-40c0-99c3-f52242743bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:43.052940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 233/457] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:43.282157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a4c-4a3d-4da1-b964-a7b9b3db36e5 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.282811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a4c-4a3d-4da1-b964-a7b9b3db36e5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.283319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a4c-4a3d-4da1-b964-a7b9b3db36e5 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.285158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 225/458] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:43.290987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72205-6994-4e3f-bd86-aa48d0618745 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.291571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72205-6994-4e3f-bd86-aa48d0618745 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.292061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72205-6994-4e3f-bd86-aa48d0618745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.315098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7fc72205-6994-4e3f-bd86-aa48d0618745 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:43.316629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7fc72205-6994-4e3f-bd86-aa48d0618745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:43.336384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:43.414547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 234/459] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:43.463384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55676a4c-4a3d-4da1-b964-a7b9b3db36e5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:43.479870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e266-daca-4a49-9f54-80baa373dd26 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.480458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e266-daca-4a49-9f54-80baa373dd26 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.480907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e266-daca-4a49-9f54-80baa373dd26 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.482688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 235/460] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:43.487693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59262f-e34a-4901-a2c2-e04b87ba2195 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.488182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59262f-e34a-4901-a2c2-e04b87ba2195 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.488628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59262f-e34a-4901-a2c2-e04b87ba2195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.523022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9b59262f-e34a-4901-a2c2-e04b87ba2195 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:43.524209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9b59262f-e34a-4901-a2c2-e04b87ba2195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:43.608411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55676a4c-4a3d-4da1-b964-a7b9b3db36e5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:43.652969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:43.674349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 236/461] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:43.684283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:43.684932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.685208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.685471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.700986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:43.702690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:43.711625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d053c8-02a6-4407-994f-3441da58cf72 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.712206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d053c8-02a6-4407-994f-3441da58cf72 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.712676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d053c8-02a6-4407-994f-3441da58cf72 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.714473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 237/462] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:43.720279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.720834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.721314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:43.728463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:43.729383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:43.730240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:43.730240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:43.730240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:43.732287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 226/463] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:43.739803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:43.753060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:43.754516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:43.841690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 238/464] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:43.842912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:43.854640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:43.917279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b020ce-78ce-4b67-9544-5ba7c0b4bf25 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:43.955951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e427f-4e6c-45d4-bc8a-bff65dcc4890 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:43.975874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840171-6e60-438b-81ad-5f726381734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:43.977147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840171-6e60-438b-81ad-5f726381734b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:43.977626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840171-6e60-438b-81ad-5f726381734b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:43.978142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840171-6e60-438b-81ad-5f726381734b None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:43.979036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38840171-6e60-438b-81ad-5f726381734b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:43.979536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38840171-6e60-438b-81ad-5f726381734b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:43.981105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38840171-6e60-438b-81ad-5f726381734b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:43.994163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 227/465] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:43.995329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840171-6e60-438b-81ad-5f726381734b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.002669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.003350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.003625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.003893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.017055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.018926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.037961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:44.038869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:44.039994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:44.040844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:44.040844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.040844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:44.040844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 239/466] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:44.041074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.118629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840171-6e60-438b-81ad-5f726381734b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.119651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.159730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.160954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.161482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.162282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.170729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bbc4-caee-4fff-a520-c547934d8def None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.173338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.174921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:44.191272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:44.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:44.193023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:44.193023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.193023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:44.193023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 228/467] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:44.193239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.211244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.211863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.212168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.212406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.213131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.214562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.247097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 240/468] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:44 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:44.247097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.302051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.322388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d71e9-6a28-4d3b-ab24-4792d249dc9d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.375891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.377335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.377687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.378257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.378929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:44.379534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.381756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.389711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166519-5d17-48f5-91b3-07f84464a3fa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.404742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 229/469] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:44.404742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.416161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.443864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.444436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.444719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.444989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.457747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.458725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:44.482478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:44.483486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:44.485021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:44.485021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:44.485021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:44.485021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 241/470] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:44.485021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.549912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720579d6-a13d-4125-9a9d-de505fc04daf None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.598445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f5224-6436-47b8-a559-8e3fc3a09246 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.600352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.602071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.602648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.603123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.603965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:44.604439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.606603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.622385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 230/471] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:44.623273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.633030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.637866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.638758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.639060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.639338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.640255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.642563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc6216-5a5f-4de7-872f-0fee7cd069fb None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.669819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 242/472] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:44 2022] POST /identity/v3/users => generated 248 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:44.672305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade217-59a1-44bf-931d-f076db3e56f1 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.672605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade217-59a1-44bf-931d-f076db3e56f1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.672882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccade217-59a1-44bf-931d-f076db3e56f1 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.674654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 243/473] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:44.681169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccade217-59a1-44bf-931d-f076db3e56f1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.718557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.760527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c8a1-d1a0-4210-a5c3-93c17f5a2b8b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.803057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.804218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.804704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.805174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.806232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:44.806846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.807691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccade217-59a1-44bf-931d-f076db3e56f1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:44.808912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e4323-ad2a-40a1-b9c7-225db31170fa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.823630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 231/474] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:44.825290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.825793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.826279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.846283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:44.847702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:44.852359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:44.852973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:44.853247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:44.853512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:44.854151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:44.855838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:44.881260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 244/475] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:44 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:44.881916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:44.929241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:44.942796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 232/476] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:44.985642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:45.009377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed828ba-d565-4162-aec4-323e104aa7fc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:45.056698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:45.057330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:45.057605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:45.057870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:45.058519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:45.059993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:45.088667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 245/477] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:44 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:45.255743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:45.376097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f925d-11ca-406b-931c-326bc171ce48 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:45.423285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:45.423907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:45.424186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:45.424459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:45.425131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:45.426506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:45.436105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.sql.core [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:42:45.436105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:42:45.436105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [parameters: {'id': '61610fa328a6431695a3dbc3c35005dc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:42:45.436105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105788) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result.read() Aug 30 00:42:45.454327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:42:45.455310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application yield resource Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise self.value Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application session.commit() Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:42:45.456254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:42:45.457108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:42:45.457961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result.read() Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:42:45.458761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [parameters: {'id': '61610fa328a6431695a3dbc3c35005dc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:45.461595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:42:45.463646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:45.465563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 233/478] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 30 00:42:45 2022] POST /identity/v3/roles => generated 144 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:45.465703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:45.568741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b267f-77d5-486d-b6f2-7978398698ea None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:45.603601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:45.604262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:45.604537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:45.604801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:45.607742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:45.617905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-bac84982-2aad-4da6-a2cf-37a00e93727e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:45.622861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:45.624936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:45.626193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 246/479] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:45.630508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:45.702758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:45.746266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de36077-5dcd-4b37-834a-5dd4cf256c88 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:45.782141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:45.782755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:45.783103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:45.783374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:45.784076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:45.784375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:45.785748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463e3d8-1633-4264-8ade-7e9ae30a934c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:45.794651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 234/480] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:46.584461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7146-75e2-4779-9aa8-3f4ba81f5f31 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:46.584986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7146-75e2-4779-9aa8-3f4ba81f5f31 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:46.584986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7146-75e2-4779-9aa8-3f4ba81f5f31 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:46.586721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 247/481] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:46.592457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:46.593241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:46.593241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:46.617183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:46.618381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:46.723405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 235/482] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:47.180096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b0f9-5123-4389-bc97-3cc1dca83072 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.180498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b0f9-5123-4389-bc97-3cc1dca83072 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.180695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b0f9-5123-4389-bc97-3cc1dca83072 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.182162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 248/483] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:47.183078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.187188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216d3ff-91b0-46b3-a0c3-d15421575fd1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.187475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216d3ff-91b0-46b3-a0c3-d15421575fd1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.187769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216d3ff-91b0-46b3-a0c3-d15421575fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.222569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7216d3ff-91b0-46b3-a0c3-d15421575fd1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:47.224261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7216d3ff-91b0-46b3-a0c3-d15421575fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:47.272738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:47.303125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 249/484] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:47.304394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559487-5a03-470f-803f-92bcddc44626 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.304693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559487-5a03-470f-803f-92bcddc44626 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.304933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559487-5a03-470f-803f-92bcddc44626 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.306162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 250/485] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:47.311455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.311863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.312191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.330942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:47.332183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:47.341564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29f0ec-25f0-4e8f-9c6d-abe23466ddab None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:47.414012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:47.414658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.414933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.415236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.436243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:47.441419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:47.442275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:47.443103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:47.444252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 236/486] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:47.444908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 251/487] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:47.450396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.586591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfe5ff-194a-46bb-aa84-67222aff24eb None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:47.601074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:47.636582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:47.637198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.637476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.637747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.638405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:47.638700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:47.640227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:47.652894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 237/488] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:47.658342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.694938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.746751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:47.811845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adf6ea-6d51-4320-ae6f-d6a66b162f94 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:47.840341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772974-cf3f-475f-aa68-3f14fdaa3298 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:47.856048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:47.856959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.857374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.857782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.869265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:47.877202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:47.878242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:47.879135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:47.880222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:47.880945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:47.881289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:47.881777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:47.882080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 238/489] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:47.883053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.893575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:47.895184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:47.912216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:47.913169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:47.914005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:47.914005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:47.914005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:47.914005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 252/490] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:47.914905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:47.922933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:47.998032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929591f1-43ef-4fa7-a3eb-8cc212db1ff1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.022262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cb880-965a-4a78-87ab-e962ef50f7b8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.032266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.034608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.035212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.035487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.035908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.046737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:48.052211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:48.053087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:48.053919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 239/491] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:48.054587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.065665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.066525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.066525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.066811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.067507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.067806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.069187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727125-ab17-4e56-84eb-18119714fc75 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.077465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 253/492] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.078675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1206-7cba-44b4-bb6f-3c57ac45b651 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.078951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1206-7cba-44b4-bb6f-3c57ac45b651 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.079244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1206-7cba-44b4-bb6f-3c57ac45b651 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.080584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 254/493] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:48.081122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1206-7cba-44b4-bb6f-3c57ac45b651 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.166337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.174624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13f71d-85f2-43e5-95d2-086a1f38be6e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.187483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1206-7cba-44b4-bb6f-3c57ac45b651 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.210670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.211312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.211588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.211857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.212593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.212894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.214383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.221421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.222533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.222817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.223114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.223784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.224079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.225884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284552d3-439f-4797-b914-3c84f71c980a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.232046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 240/494] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.232929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.239065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 255/495] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.239065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.239065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.239065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.255081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:48.256284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:48.287989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.333348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 256/496] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:48.342144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.351396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73dde2-5906-49fa-8592-d5337208198d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.391214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.391839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.392117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.395026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.404069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.405613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:48.424798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:48.425707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:48.426553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:48.426553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:48.426553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:48.431176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 241/497] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:48.432025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.442278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.446615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e62a64-1935-4b8d-9930-e27217475150 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.482445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.483107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.483470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.483800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.491077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.492555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-d2dd7bb0-cc75-4e17-92ff-ce23d9ccc304 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:48.506567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:48.507644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:48.508757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 257/498] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:48.509315 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fddf87-565c-46d4-b535-b3fde4dbf1f4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.509381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fddf87-565c-46d4-b535-b3fde4dbf1f4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.509626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fddf87-565c-46d4-b535-b3fde4dbf1f4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.515044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 258/499] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:48.515044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fddf87-565c-46d4-b535-b3fde4dbf1f4 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.544834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.549596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1c4bc-ae17-48a4-8129-5de0c6af222e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.584916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.585548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.585830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.586106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.599082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.600575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:48.616936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:48.618336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:48.619237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:48.619237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:48.619237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:48.619237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 242/500] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:48.619237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.634439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fddf87-565c-46d4-b535-b3fde4dbf1f4 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.668392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.669005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.669259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.669532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.670174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.670479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.672308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb10ad7-dc3f-4283-a005-565a4b36ff74 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.680134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.685665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 259/501] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.687087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.687378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.687723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.705358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:48.706582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:48.723980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fb3a5-40e8-406e-a779-e91c7754e5ee None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.757241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.757772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.758054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.758329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.760854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.761126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.762577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.776817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 260/502] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:48.777665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.792341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.811059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 243/503] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.811921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.891114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183ec35-61d6-41cd-bf2c-7a87ac57d5a3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.927491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27486-3622-4961-a175-b678c1467456 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:48.929066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.929820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.930107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.930381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.940972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:48.945935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:48.946806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:48.947727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:48.948513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 261/504] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:48.949190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:48.971762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:48.972379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:48.972665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:48.972937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:48.973586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:48.973893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:48.975320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:48.991035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 244/505] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:48.991035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.029026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.061295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180fe-e4f8-4991-9552-9e4b81efbf98 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.091211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29ac3-6c78-44e8-a90f-1a11ff3550b1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.094840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.095499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.095774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.096042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.103259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.104730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:49.116868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:49.117858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 262/506] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:49.118771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.130191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.130800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.131110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.131387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.180228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.205803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.207325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.234176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea02b0-98a5-4393-9589-1edce59b60db None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.262592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.268381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.269022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.269308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.269579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.270250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:49.270553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.272002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.280319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 263/507] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:49.281086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.311254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 245/508] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:49.312174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.383074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfb6f-7518-4256-83c3-1186861fff1a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.391450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.419052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de405-7edc-4e40-a768-0de5ee82f06a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.423774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.424538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.424817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.425081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] PATH_INFO: `/v3/users/glance` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.433639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.435181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:49.446970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:49.447914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:49.448807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 264/509] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:49.449371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.457712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.458416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.458755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.459137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.471100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.472564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:49.486976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:49.488104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:49.489116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:49.489116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.489116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:49.489116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 246/510] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:49.489116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.511867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.559485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31deeef-84ac-4493-88d4-a8e0bc118d70 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.596678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5ee5-712f-4c77-b6b5-44801646575b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.598934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.599603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.599881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.600157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.600891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:49.601200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.602674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.622978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.631414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.632069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.632352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.632625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.639501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 265/511] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:49.640420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.643872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.645623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:49.660099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:49.660991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:49.661826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:49.661826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.661826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:49.661826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 247/512] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:49.662034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.740840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0219e-f488-4feb-8a0b-0f6ea44bfcc5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.753631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.762443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9061c-447d-4877-99ad-c25fff11c4d7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.775695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.776330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.776608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.776880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.786997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.788503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.795541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.796251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.796530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.796800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.797522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:49.797825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.799294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:49.804433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:49.805586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:49.806446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:49.806446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:49.806446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:49.806446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 266/513] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:49.806446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.823109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 248/514] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:49.823897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.867188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:49.908022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63d363-c4d0-43f2-bb62-fe7550fd4991 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.930509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49449-c9c4-4a08-a9fe-661bd422c8be None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:49.947173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.948046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.948323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.948590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.949403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:49.949709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:49.953330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:49.966888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 267/515] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:49.967925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:49.972624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:49.973517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:49.973886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:49.974248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:49.994613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.047804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.049480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.076044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909757-a390-4bac-81ad-5423f2ea7f67 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.091362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.116922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e096c-276c-4c04-9144-230336417aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.117743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e096c-276c-4c04-9144-230336417aab None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.118123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e096c-276c-4c04-9144-230336417aab None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.118496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e096c-276c-4c04-9144-230336417aab None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.119301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e096c-276c-4c04-9144-230336417aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:50.119742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e096c-276c-4c04-9144-230336417aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.121358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e096c-276c-4c04-9144-230336417aab None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.133346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 268/516] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:50.134199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e096c-276c-4c04-9144-230336417aab None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.162550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 249/517] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:50.162550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.231610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.251629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e096c-276c-4c04-9144-230336417aab None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.263841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094243-1356-4609-84d6-8ca6179236c1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.285617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.286219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.286492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.286764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.287428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:50.287726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.289103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.299626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.300239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.300517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.300744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 269/518] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:50.300859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.301530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.309235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.372142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.373655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.406161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704874-b645-43b3-a453-2282ca56fedd None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.440819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.441440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.441713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.442005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.442653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.444155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.470652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.473326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 270/519] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:42:50 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:42:50.474449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.481933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 250/520] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:50.482817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.538565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.606619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dcc80-19cd-4ad0-8c83-c9f11ec5dd2b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.610896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862df596-9de6-44b8-a633-39cb9905d0b1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.644400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.644940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.645220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.645491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.651735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.652925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.653672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.653948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.654194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.656094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.666879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.668479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:50.669700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:50.670544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:50.671707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 271/521] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:50.672751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:50.686949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:50.687931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:50.688804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:50.688804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:50.688804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:50.688804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 251/522] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:50.689461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.740071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.783913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4429050-b556-45cf-bbfa-0317101152bf None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.792930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:50.804225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ed1b2-2898-44fc-b6bd-91fd211c8669 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.825273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.826041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.826320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.826587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.838096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.839862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.844052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:50.845261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:50.845544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:50.845820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:50.856201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:50.857423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 272/523] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:50.858264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.866179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:50.867776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:50.883512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:50.884509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:50.885501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:50.885501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:50.885501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:50.885501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 252/524] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:50.885501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:50.987112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde65a4-2063-48af-942c-00e2ec74c204 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:50.994042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e6e82-a11e-4516-8bcc-5fac642313d0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.008679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.024158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.024772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.025046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.025326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.026084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.026393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.027884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.036007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.036911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.037257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.037534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.038178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.038479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.039942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.050763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 273/525] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.050763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.061741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 253/526] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.061741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.063282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.151385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e161-b8b7-485a-a778-aece11fab48f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.169180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff7d0c-7ebe-4d98-9758-0f91c49ff619 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.197633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.198514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.198975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.199414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.200261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.200704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.202675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.206434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.207064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.207344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.207617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.208251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.208550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.209900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.224211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 274/527] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.229242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 254/528] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.229242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.230328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.283095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.306097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.348514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41968-08c6-4d5d-9295-7104d01959d0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.350030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb6519-2fef-4eef-9d0c-5301606a6a69 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.383313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.383946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.384225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.384498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.385876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.386602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.386874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.387283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.391744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.393227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:51.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:51.406795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 255/529] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:51.408123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.468481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.470320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.509871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.519887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f198c6-60bd-4e84-8e1b-a34cd80e0449 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.545014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.558766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.559687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.559967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.560244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.567693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.569190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.581146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 275/530] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:51.581146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:51.587222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:51.588061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:51.588910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 256/531] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:51.589502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.690895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b29b9-c0ea-48d5-ba18-1d29989a9b2a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.702880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817c9c-22f8-430e-abcb-fc8d87eb6934 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.726952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.727664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.727958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.728228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.728966 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.729275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.730775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.740067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.740726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.741009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.741289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.742003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.742316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.743858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.747490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 276/532] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.749634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.753169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.765351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 257/533] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.766667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.815278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:51.887266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de0db0-21e7-4df7-bdb3-2ff9f74ed8ca None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.905019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce11c2c-3313-464a-a81c-a92c3b8c3947 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:51.925622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e1991-10dd-4841-8845-be601569277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.926252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1991-10dd-4841-8845-be601569277e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.926536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1991-10dd-4841-8845-be601569277e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.926815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1991-10dd-4841-8845-be601569277e None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.927571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e1991-10dd-4841-8845-be601569277e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:51.927893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e1991-10dd-4841-8845-be601569277e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.929356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e1991-10dd-4841-8845-be601569277e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.948579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 258/534] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:51.949534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e1991-10dd-4841-8845-be601569277e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:51.951389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:51.952260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:51.952713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:51.952987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:51.967593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:51.969656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:51.989473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:51.990425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:51.991289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:51.991289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:51.991289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:51.995107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 277/535] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:51.996181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:52.058754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:52.086564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e1991-10dd-4841-8845-be601569277e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:52.108139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5321ca-7063-49be-8cf8-05fe7db801e8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:52.111341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:52.136147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:52.137803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.138423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.138776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.140020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=87af1d19163043ffaa3412487446b02f) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:52.140377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:52.143129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:52.146020 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:52.146720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.147115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.147486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.148418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=97d15d2823be4939a617603cfdad0aef) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:52.148906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:52.150730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 259/536] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/role_assignments?user.id=87af1d19163043ffaa3412487446b02f&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:52.151796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:52.159390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:52.167891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 278/537] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:51 2022] GET /identity/v3/role_assignments?user.id=97d15d2823be4939a617603cfdad0aef&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:52.297263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff415-42d2-4255-bc96-e4b82470a381 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:52.352402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:52.352998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.353348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.353622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.354282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:52.354653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:52.356092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36295790-1f60-41fa-97f7-23edf4d1eb7a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:52.370316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 260/538] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:52.376365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:52.499524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0852-6f76-440f-974a-67362279de58 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.499854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0852-6f76-440f-974a-67362279de58 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.500053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0852-6f76-440f-974a-67362279de58 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.500782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57512f-7911-43e8-911a-fe58d569e31d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:52.501752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 261/539] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:52.507411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.507737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.508015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.527545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:52.530764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:52.531987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:52.563463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:52.583839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:52.584494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.584779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.585050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.600188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:52.602709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:52.614985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 262/540] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:52.629121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:52.630189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:52.631277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:52.639875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 279/541] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:52.645558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:52.812304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc56e2-dc9f-4c9b-ac2c-d56e212d3124 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:52.865516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:52.866049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:52.866322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:52.866589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:52.867325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:52.867716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:52.869189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:52.888168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 263/542] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:52.895199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:52.983030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:53.019544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:53.073772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31be59-a61f-47af-aa4d-ee317a1b1b73 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:53.155407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:53.155948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:53.156227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:53.156498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:53.157283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=3acf83ba003e4e02a5ddd87c2714792d) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:53.157562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:53.159182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:53.173606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 280/543] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:52 2022] GET /identity/v3/role_assignments?user.id=3acf83ba003e4e02a5ddd87c2714792d&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:53.177538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b81849-293f-4fcf-87cf-3b62978db5f2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:53.231156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:53.231820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:53.232101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:53.232372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:53.249143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:53.254964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:53.255937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:53.256756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 264/544] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:53.262606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:53.322224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:53.416631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219b18-752d-4f3d-8991-c109382ff09b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:53.462534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:53.463191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:53.463525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:53.463803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:53.464491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:53.464795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:53.466190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:53.476197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 281/545] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:53.481618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:53.587449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:53.619327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca07036-a4a9-4511-9c8d-48ffe926410b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:53.663117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:53.663782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:53.664063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:53.664338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:53.680974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:53.682374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:53.701689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:53.702922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:53.703799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:53.703799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:53.703799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:53.707374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 265/546] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:53.709150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:53.820335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ffaf-0b69-48d9-bdc1-d5145b217172 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:53.822729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:53.875247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:53.876186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:53.876468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:53.876742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:53.877407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:53.877706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:53.879114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2fe6b-4b5b-4969-85c0-2193cacdd5ac None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:53.892550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 282/547] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:53.897813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:54.046368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212ef0-6e71-4629-9ba2-7914c8712a11 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:54.088786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:54.097063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ad9d-c84b-4c55-9367-ee6678269f6b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.097602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ad9d-c84b-4c55-9367-ee6678269f6b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.097602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ad9d-c84b-4c55-9367-ee6678269f6b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.098950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 283/548] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:54.099431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:54.100067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.100342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.100612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.108639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.108919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.109188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.112212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:54.113945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:54.127867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:54.129314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:42:54.130367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:54.135107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 266/549] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:42:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:54.138241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:54.148584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:54.149982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:54.279978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 284/550] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:42:54.301920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871795-0075-45a5-a258-0d29862ef477 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:54.349263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:54.350122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.350561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.350986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.351923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:54.352363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:54.353970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256f1c5-3b2c-420a-9d32-3e92380a76ac None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:54.375379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 267/551] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:42:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:54.378657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:54.511606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:54.518395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f831-90c7-4610-9462-517d34d8fabc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:54.561616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:54.562194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.562472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.562737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.602226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd670575-0930-4063-ac64-7d4807477993 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.602843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd670575-0930-4063-ac64-7d4807477993 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.603364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd670575-0930-4063-ac64-7d4807477993 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.605191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 268/552] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:42:54.610045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5719c-6e43-4858-9d0e-c055dc6f7923 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.610440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5719c-6e43-4858-9d0e-c055dc6f7923 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.610826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5719c-6e43-4858-9d0e-c055dc6f7923 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.614529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 269/553] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:54.615709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5294-ae46-40e2-a1b4-be8f3fae0204 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.615982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5294-ae46-40e2-a1b4-be8f3fae0204 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.616257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5294-ae46-40e2-a1b4-be8f3fae0204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.643923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-140e5294-ae46-40e2-a1b4-be8f3fae0204 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:54.646253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-140e5294-ae46-40e2-a1b4-be8f3fae0204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:54.652359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:54.653930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:54.696728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:54.722981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 270/554] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:54.724766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.725048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.725318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.755225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:54.756229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:54.758020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:54.763398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 285/555] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:42:54.764309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:54.846189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 271/556] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:54.847189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:54.894115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07abf1-ed3e-4403-8780-5288fae4150f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:54.967729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:54.968780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:54.969565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:54.969739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956cfba-8381-423b-afde-4e9b8215b5ac None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:54.970483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:54.978002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:54.995841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:55.000845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.001734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:55.002552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 286/557] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.009650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.010979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.011959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.012467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.012928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.024323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.026071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:55.042200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.043181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:55.044048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:55.044048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.044048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:55.046137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 272/558] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.047137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.076557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.122645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06717d-c2d1-4b49-864e-c6e990476433 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.157889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dfd75-c17d-4edf-8d9e-98e8b55ddf9b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.158085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.158549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.158825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.159142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.159815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:55.160114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.162463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.170765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 287/559] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:42:55.171779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.197400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.198040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.198313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.198619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.209189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:55.214322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.215267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 273/560] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.216740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.237577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.284499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b8e83-6978-488c-a8ea-94d80e600dca None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.301381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.321011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.321620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.321894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.322163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.322803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:55.323126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.324496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.329200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86b04-35e3-4302-9871-87fd878cc33c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.339259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 288/561] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:55.340883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.365248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.365903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.366185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.366455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.377919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.379356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:55.394804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.395813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:55.396689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:55.396689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.396689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:55.396689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 274/562] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.396910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.456030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d42c3-0cd4-4aa5-82f3-2ba0c9148df6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.476707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.492037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.492696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.492964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.493235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.504252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:55.509124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.510043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:55.510863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:55.511541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153404a2-2c8c-440d-8747-45ae0fad7090 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.515300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 289/563] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.516583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.547574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.548093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.548368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.548687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.549282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:55.549583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.550997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.555781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.565306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 275/564] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:55.565306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.634645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31290f11-3ec3-450d-a805-483b9c7ddb41 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.678727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.679565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.679841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.680154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.682668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b025c-85d5-4ea6-a258-2f1c5fcd4dea None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.688425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.689908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:55.705359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.706339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 290/565] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.712045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.724010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.730738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.731386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.731668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.731942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.732588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:55.732888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.734543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.746875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 276/566] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:55.747837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.807928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.815163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fb490-a021-427b-9742-4d568ea07cb0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.846645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb030-f358-4cc3-88ed-def973610e4b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:55.853673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.854429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.854809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.855179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.855949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:55.856251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.857648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47264-aaf2-42af-a6ec-0977c65ccd44 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.866406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 291/567] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:55.867438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca11ed-9387-43b8-bebd-63e0e516caca None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.867714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca11ed-9387-43b8-bebd-63e0e516caca None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.868030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca11ed-9387-43b8-bebd-63e0e516caca None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.869195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 292/568] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:55.869881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca11ed-9387-43b8-bebd-63e0e516caca None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.884256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:55.884941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:55.885158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:55.885421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:55.895856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:55.897805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:55.912714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:55.920145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:55.921943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:55.921943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:55.921943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:55.921943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 277/569] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:55.921943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:55.966625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:55.980375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca11ed-9387-43b8-bebd-63e0e516caca None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.015399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.016407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.016884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.017329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.018567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:56.019108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.021683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.023806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b1c3-2bf0-451c-9624-d000da918678 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.036831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 293/570] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:56.037758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.057199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.070455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.071115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.071395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.071662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.080528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.081961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-c236e0de-1be7-4b41-81ad-113b955a9136 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:56.094679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.095861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:56.097028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 278/571] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:42:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.099257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.099580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.099867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.117556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:56.118798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:56.148217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaeaee-22dc-4a8f-81aa-b213888a9dae None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.184692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.185309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.185585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.185899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.187900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 279/572] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:56.188733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.196143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.197587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:56.213051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.214003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:56.214888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:56.214888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:56.214888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:56.214888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 294/573] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.215144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.228429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.283889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.301365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c94f3-bbe9-4ae4-9a10-70bdde20b93a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.315340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fca6a2-5017-44d6-ba37-ef5b56d21c39 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.335848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.336451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.336746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.337128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.337847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:56.338149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.339591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.348580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.349183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.349462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.349731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.350456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b98ce88b48c34731a1eae85d23de837a) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:56.350763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.352243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.352985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 280/574] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:56.354059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.362275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 295/575] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/role_assignments?user.id=b98ce88b48c34731a1eae85d23de837a&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:56.363366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.490923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.497827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9924e-f576-4e16-8df3-4e00d694744e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.510627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c7ac-c278-4fec-bb2f-97e34db86236 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.538724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.539425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.539759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.540102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.540838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:56.541134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.542577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.547162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.547767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.548039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.548307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.548940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:56.549237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.550602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.552320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.564357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 281/576] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:56.564544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 296/577] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:56.565574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.570251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.670385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e7987-577b-4830-ac48-7af18addcad8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.685013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e9d6-a62b-4b86-baf3-ac60d2eb20e2 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.706871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.707569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.707838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.708103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.719403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:56.723462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:56.724120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.725279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:56.726296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 282/578] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.727060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.727335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.727744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.728133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.740840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.742773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.745666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:56.756038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.757249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:42:56.758537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:56.761960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 297/579] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.762823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.820530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:56.848104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2902f-110e-41c3-ad42-4e994774b9d8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.870936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f96dbc-46b1-45e0-bd69-6e8ff291977a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:56.886401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.887068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.887346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.887610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.898254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.899739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.905667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:56.906293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:56.906569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:56.906835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] PATH_INFO: `/v3/users/glance` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:56.914725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:56.915639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.916551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:56.917491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:56.917491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:56.917491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:56.917657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:56.917726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 283/580] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.917726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:56.931973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:56.933154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:42:56.934336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:56.938114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 298/581] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:56.938923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:56.989742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.032153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f263e0-b63b-4d58-8a38-1f9085da62e4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.047425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea0e14-103e-4590-818e-0fd11eb9f810 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.061364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.070050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.071107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.071528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.072239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.073312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.073908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.076083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.082316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.082937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.083245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.083518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.084286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.084591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.086032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.086793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 284/582] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.087913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.105326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 299/583] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:42:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.106921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.196271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe8ffa-86ce-44d4-bf9f-dda1f061b403 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.211542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fffa8-7d5b-4e30-b99c-3aa95b0b9ee5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.223325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.233381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.234126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.234509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.234940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.235722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.236079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.237460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.253853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 285/584] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.253853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.255798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.256419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.256692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.256964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.257754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.258057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.260058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.279544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 300/585] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.280571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.313348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.359638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92f7c3-2858-4116-83d3-a72335bd3d8b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.391188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcfe49-56c4-4a65-83c1-d6f3113a2e04 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.399324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.399964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.400236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.400504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.413738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.415366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.433456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.434101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.434382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:57.434482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:57.435393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:57.436422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:57.436422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.436422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:57.436592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.436663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 286/586] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:57.436663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.453829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.455295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:57.473850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:57.474886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:57.475819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:57.475819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.475819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:57.475819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 301/587] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:57.475819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.491740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.556881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785c962-c1a7-4d7e-a971-763fe35e9853 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.566526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.587364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172934cd-1fa5-4368-be5c-6b6c42117eaf None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.591615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.592429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.592842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.593180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.600462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.601986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:57.614236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:57.615393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:57.616495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:57.620117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 287/588] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:57.620963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.627292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.627923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.628195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.628461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.639705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.641413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:57.656006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:57.656945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:57.657867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:57.657867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:57.657867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:57.657867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 302/589] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:57.658604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.728767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.733908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7403e-1def-488a-a7aa-9786307e9d97 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.769102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64dc87-07e9-4abc-b663-3c5fb3ea921f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.771243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.771909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.772193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.772474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.773143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.773445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.774819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.788702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 288/590] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.789536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.806137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.806973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.807421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.807858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.808638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.809081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.810758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.812666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.824771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 303/591] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.825592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.904948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d3166-c193-48bb-b8ab-c4a73da5039f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.941735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957dc15-62d4-4e8e-9b5f-58a7cf4bf176 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:57.942363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.942995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.943299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.943603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.944269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.944575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.946036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:57.959663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 289/592] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:57.960576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:57.982547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:57.991048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:57.991659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:57.991937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:57.992210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:57.992850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:57.993148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:57.994763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.011468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 304/593] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:58.011468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.070777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.076335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d47af-a94e-435e-a414-92ac5ab58094 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.112953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.113780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.114158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.114469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.120500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf63c76-3f8d-4151-9cca-0bc69f850a74 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.122460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.124076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:42:58.135721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:58.136917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:42:58.138088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:58.139126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 290/594] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:58.139972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.166220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.166932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.167310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.167588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.174688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.176365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:58.189006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:58.190091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:58.191184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:58.192080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 305/595] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:58.193560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.228471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.253765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe37ea-9c2d-4497-8af3-cf10d2bad3c5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.288539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.289203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.289481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.289760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.298000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bebf5-0401-41f1-9336-2bf8a2795d7e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.304242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=28f8f65dc678403081189e31ef5f8a10, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.305703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.311879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.319083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 291/596] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:58 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:58.320204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.336047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.336804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.337160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.337498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.346510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.348183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:58.361135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:58.362300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:58.363458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:58.369109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 306/597] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:58.369109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.445217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f2487-6ba9-4b0a-8f67-2f1ccadf1494 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.477951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.483414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7258bc5-fb4d-44fc-902a-c57bd75f7c6d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.488532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.489156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.489429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.489699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.490432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:58.490736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.492201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b05f1e-f33b-4f44-a095-b2a09f47ccf5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.510347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 292/598] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:58.511404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96b85-3044-4066-8dd9-1adf9f49e2b0 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.511686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96b85-3044-4066-8dd9-1adf9f49e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.511958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b96b85-3044-4066-8dd9-1adf9f49e2b0 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.517316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 293/599] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:42:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:42:58.517316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b96b85-3044-4066-8dd9-1adf9f49e2b0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.528333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.529026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.529476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.529752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.530859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:58.531198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.532647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.545945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 307/600] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:58.546776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.584608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.627762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b96b85-3044-4066-8dd9-1adf9f49e2b0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.655229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8ee4-1a0b-4529-9f47-1f430d61441e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.662300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.662981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.663350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.663681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.664407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:58.664709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.666154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509736-c6f3-4935-ad3d-7e1be8e123c8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.678410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 294/601] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:58.679836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.680221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.680596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.698852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:42:58.700171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:42:58.701631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.702696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.703072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.704049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.714488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.716059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:58.732260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:58.733313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:58.734278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:58.734278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:58.734278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:58.734278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 308/602] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:58.734501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.750880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.788432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 295/603] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:42:58.789182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:58.823888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:58.842940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906e7d2-694f-4a3e-93f1-5cc63fb82547 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.883130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:58.884243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:58.884775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:58.885233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/87af1d19163043ffaa3412487446b02f/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:58.901746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=87af1d19163043ffaa3412487446b02f, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:58.904615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:58.927106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d20d-768d-423e-8abd-08ff8e9bec02 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:58.927554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 309/604] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:58 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/87af1d19163043ffaa3412487446b02f/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:58.929548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.003662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.004604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.005026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.005470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/97d15d2823be4939a617603cfdad0aef/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.039579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=97d15d2823be4939a617603cfdad0aef, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.041343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.066776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 296/605] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:58 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/97d15d2823be4939a617603cfdad0aef/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:59.071662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.117998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880829b-c2e6-4745-a0c1-f90a44b64321 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.134821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.163999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.164625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.164902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.165174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.165827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:59.166123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.168622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.178183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.184589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 310/606] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:59.193913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.235309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc494-b4fc-42ac-b7d8-f8efd945b402 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.284284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.285260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.285687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.286094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.301009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:42:59.307448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:59.308386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:42:59.309354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:59.309988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced24fda-a331-4203-9cc5-0aad231865c9 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.310390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 297/607] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:59.319680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.362714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.363773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.364326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.364828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.380773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.382520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:42:59.395736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:59.396693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:42:59.397585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:42:59.400008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 311/608] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:42:59.406547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.475874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.512439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.532353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1346f3c-77a2-4559-bcbb-3961966596d3 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.553247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d96da-44ed-4737-9a40-737c8f4d6772 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.573259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.574250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.574721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.575230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.576770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:59.577398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.579429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.599754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 312/609] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:59.606723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.625362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.625989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.626274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.626554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.628896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:42:59.628896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.629306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.655854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 298/610] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:42:59.657934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:42:59.768572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736369b-3b66-4b80-a8e8-7b25f1718e76 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.786284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.795149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b662b-5851-4ea8-a1e8-7dc80f76826b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:42:59.806064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.806681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.807193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.807300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/3acf83ba003e4e02a5ddd87c2714792d/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.823337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=3acf83ba003e4e02a5ddd87c2714792d, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.824857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.844951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 313/611] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:42:59 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/3acf83ba003e4e02a5ddd87c2714792d/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:42:59.846131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:42:59.846788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:42:59.847102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:42:59.847385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:42:59.860950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:42:59.862552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:42:59.884480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:42:59.891414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:42:59.892502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:42:59.893839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:42:59.893839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:42:59.893839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:42:59.893839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 299/612] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:42:59.901089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:00.026013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a482f-e180-4922-a0d9-ab9b54bd3f52 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:00.063623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:00.064224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.064508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.064785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.065443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:00.065747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:00.067164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:00.080053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 314/613] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:00.085952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:00.208130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378e0d8-7984-4dbe-b75c-84c19b1852cf None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:00.243558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:00.244193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.244466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.244736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.252766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:00.254214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-1a2898d2-54b4-4480-8e78-8d325d7766d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:00.268243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:00.269218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:00.270548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 300/614] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:00.277055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:00.329567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:00.446930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96808bfc-94cb-47ba-8356-048cc9710a5e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:00.490286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:00.490888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.491221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.491559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.492290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:00.492590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:00.494017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:00.506051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4bd6-e5e1-4424-a93a-958d5b26c821 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.506353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4bd6-e5e1-4424-a93a-958d5b26c821 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.506599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4bd6-e5e1-4424-a93a-958d5b26c821 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.508202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 301/615] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:00.512591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 315/616] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:43:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:00.513111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.513392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.513665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.520171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:00.543412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:00.544620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:00.645620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:00.668362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 302/617] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:43:00.670438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:00.694115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75862a9-f089-41af-8ef0-defafb4941cc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:00.740756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:00.741446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.741716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.742022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.761486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:00.762938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:00.780747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:00.781806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:00.782694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:00.782694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:00.782694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:00.782694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 316/618] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:00.787837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:00.929878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68977-946e-4ed0-9b54-cf68d872198c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:00.978474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:00.979454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:00.979926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:00.980377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:00.981252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:00.981730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:00.983452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:00.996043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 303/619] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:01.003035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.124400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.192022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fdc4c-906b-4780-a2a2-30b4d24cd3da None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.240872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.259495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.260136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.260422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.260692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.267371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.276058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:01.277567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:01.305202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:01.306210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:01.307164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:01.309064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 317/620] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:01.314290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.344547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b811-c66c-465d-ab8b-5d48fb52c633 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.416425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.417078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.417357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.417630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.432235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7cd5792c-4a77-4c49-bd85-6de54efe0b60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:01.438272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:01.439380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:01.440212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 304/621] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:01.440965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f684-f2d7-4f37-b981-b8e3c1f6bd18 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.441857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f684-f2d7-4f37-b981-b8e3c1f6bd18 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.441857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7f684-f2d7-4f37-b981-b8e3c1f6bd18 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.446683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 305/622] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:01.446683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283257d-e894-4cab-854f-ea477744327f None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.446683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283257d-e894-4cab-854f-ea477744327f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.446683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283257d-e894-4cab-854f-ea477744327f None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.446683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 306/623] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:01.448697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cda81-4ff6-44be-885a-bb1b8d77bb48 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.449355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cda81-4ff6-44be-885a-bb1b8d77bb48 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.449355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cda81-4ff6-44be-885a-bb1b8d77bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.465964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816998a-5c48-4af7-b131-9debcc4de90b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.467555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f40cda81-4ff6-44be-885a-bb1b8d77bb48 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:01.469054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f40cda81-4ff6-44be-885a-bb1b8d77bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:01.504836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.505882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.506383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.506850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.508238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:01.508717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:01.510960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:01.524871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 318/624] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:01.525762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.555028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 307/625] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:01.555028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.555028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.555028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.570302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:01.572029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:01.600257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.609311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.652243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a009fd-40d0-49cd-aef1-214415f0aab6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.656017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 308/626] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:01.656402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.693176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.694222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.694738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.696395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.698888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:01.700179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:01.701882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:01.720549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 319/627] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:01.726971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.839330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de5e27-65ce-4b73-b9da-19bb0c718f8d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.879409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8482602-38dd-47d9-8ebe-8d0f73f152ac None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:01.889969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.890649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.890928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.891231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/b98ce88b48c34731a1eae85d23de837a/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.900120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.912435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=b98ce88b48c34731a1eae85d23de837a, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:01.914693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:01.922536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:01.923499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:01.923986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:01.924432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:01.931631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 309/628] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:01 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/b98ce88b48c34731a1eae85d23de837a/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:43:01.933191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:01.937458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:01.939450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:01.963122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:01.971145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:01.972127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:01.973115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:01.973115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:01.973115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:01.980783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 320/629] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:01.987697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.067111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888c85f-09d4-4bec-b42f-09ace2b46e1a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.110160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.110865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.111189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.111472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.123483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:02.128458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:02.129432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:02.130294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 310/630] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:02.131120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.148292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186c6a9-b411-4ae8-8b35-29432539d2cc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.188589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.189220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.189624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.189903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.190566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:02.190870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.192334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.211062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 321/631] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:02.211062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.245297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.255782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.281730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71244-178f-49ea-a2f5-d56c5c1fb5ea None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.330610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.331375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.331728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.332110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.344342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3fa9-d691-4952-a26f-91a286aceacc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.346983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:02.351835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:02.352821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:02.353689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 311/632] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:02.354399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.388659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.389848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.390277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.390726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.394542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:02.395063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.396769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b432c-4fbe-4282-ac92-bee0f028fe53 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.406486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 322/633] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:02.408013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7985701-0bd7-426b-a278-0798f0c03e55 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.408445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7985701-0bd7-426b-a278-0798f0c03e55 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.414896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7985701-0bd7-426b-a278-0798f0c03e55 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.416711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 323/634] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:02.417630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7985701-0bd7-426b-a278-0798f0c03e55 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.476859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac9553-90e3-4f46-8358-57a540f8cb4a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.514178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.523209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.523858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.524138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.524413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.524652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7985701-0bd7-426b-a278-0798f0c03e55 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.534588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.536746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.540302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:02.551581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:02.552574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 312/635] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:02.553514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.564087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.564700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.564977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.565249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.565923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:02.566224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.567639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89762a5-a9de-4869-9b95-66a2372643df None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.576276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 324/636] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:02.577406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.577698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.578207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.597493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:02.598705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:02.670361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561948a-29d5-4008-8c05-f3795262ac24 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.673510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 325/637] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:02.674359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.720176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.720964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.721241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.721660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.733274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.734925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:02.756005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:02.756964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:02.757918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:02.757918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:02.757918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:02.764769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 313/638] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:02.765862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.779255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.796605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54601a7-726b-48cc-957c-f6cbabcfe881 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.831121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:02.841213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.843289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.844216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.844989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.846210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:02.846744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.848611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.873155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 326/639] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:02.874809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:02.892331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997af0-c345-428d-8068-c277d7e4f81d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:02.934817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:02.935582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:02.935863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:02.936294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:02.950565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:02.952267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:02.969085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:02.970122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:02.971101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:02.971101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:02.971101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:02.971101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 314/640] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:02.972023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.007104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948224-9455-4376-bea7-acb959ea6ee7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.043047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.043589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.043881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.044138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.044805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:03.045089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.046531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.058503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 327/641] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:03.059354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.064554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.093495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e342d-8643-48e5-9cfb-eacfce6e3b20 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.097245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.133855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.134572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.134914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.135285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.168099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837781-44ae-4aff-8d2d-3d4a15f649ca None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.203954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.204571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.204849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.205120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.205765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:03.206083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.207533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.209658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.211332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.220237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 328/642] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:03.221045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.283652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.324393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533982-1b99-482a-9dd8-0bcb2e8a5683 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.324738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 315/643] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:03.326321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.330595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.367250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.367888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.368168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.368437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.378940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:03.383812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:03.384865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 329/644] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:03.385733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.447860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e958c5-45e2-460b-a626-449f842e8611 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.484674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.485317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.485698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.485972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.492528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80891-840e-45a2-a923-14f0fa203e41 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.493455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.494904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:03.506614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:03.507942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 316/645] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:03.509372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.521627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.537382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.538026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.538306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.538585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] PATH_INFO: `/v3/users/glance` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.546515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.548021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:03.560031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:03.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:03.562102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 330/646] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:03.562696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.584294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.625317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda275-6d8b-405a-92a9-1dcd7f8c5f09 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.660477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.661112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.661392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.661668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.673805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.675288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.676410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597dc57-7e9b-4071-b29c-bc77fd70d49a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:03.691970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:03.693194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:03.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:03.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:03.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:03.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 317/647] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:03.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.719360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.719974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.720258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.720530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.721194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:03.721496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.722995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.731838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 331/648] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:03.732744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.791372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.806178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b12c-6bbe-49e4-aba1-94474c229c0b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.823755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:03.844579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.845210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.845471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.845746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.846471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:03.846778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.848216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.853425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8493b-4ffa-4400-a5f5-ee7d12981218 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:03.866179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 318/649] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:03.871456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.889169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:03.889862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:03.890132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:03.890400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:03.891216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:03.891459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:03.892888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:03.911630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 332/650] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:03.912461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:03.976808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4280f9-570b-4145-9efb-2c0c01b6d3a3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.011782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbe7c8-d090-469f-accf-0689e225542f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.013324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.017072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.017704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.017987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.018273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.018914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:04.019248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.020620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.032422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 319/651] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:04.033239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.047875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.048504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.048807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.049069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.059518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.061054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.069072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-5749da7e-fda6-406c-80a4-ed596d6c3e50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:04.076983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:04.078009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:04.079029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:04.079029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.079029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:04.079029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 333/652] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:04.079429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b192-74a0-43cc-8578-148b995755a1 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.079739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b192-74a0-43cc-8578-148b995755a1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.079937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b192-74a0-43cc-8578-148b995755a1 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.081191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 334/653] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:04.081881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b192-74a0-43cc-8578-148b995755a1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.143149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e54a3-0eee-4468-80ac-792f75ec2e20 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.178000 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.178707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.178984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.179281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.184537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b192-74a0-43cc-8578-148b995755a1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.220986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.221602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.221883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.222154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.245764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.247257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.253061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.287180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.304135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.305879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.354257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 320/654] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:04.355201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.407215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 335/655] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:04.408058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.458855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055437ed-cf4a-4c8d-8077-27b1649a9f09 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.465962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.492735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.493360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.493631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.493901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.495790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.501035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.502823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.510516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9399-c84e-4980-8be0-221fc9d8f04e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-24ce95f8-6f3f-4f4b-ab4c-d2d8d5c0a0a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:04.513744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:04.514865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:04.515885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 321/656] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:04.516399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.516480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.517012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.534612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:04.535833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:04.547671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.548246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.548549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.548922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.549553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:04.549829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.551239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.562633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 336/657] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:04.563466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.615726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 322/658] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:04.616593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.676134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ec3ec-32c0-4e7c-b4e4-06fbdedf2b08 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.690925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.713723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.714755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.715246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.715713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.726977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891bb88-010d-4b99-a48a-b50142dd80ed None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.727933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.729780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:04.745450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:04.746427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:04.747464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:04.747464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.747464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:04.749198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 337/659] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:04.750247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.761273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.774759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.775456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.775728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.775984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.786402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.787861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:04.803605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:04.804568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:04.805464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:04.805464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:04.805464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:04.805464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 323/660] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:04.805464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.879724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70023709-7d05-4752-99ed-80df5145ea68 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.912697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d35bb6-2c54-44b6-9382-8e375521a739 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:04.915627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.916240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.916518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.916788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.917481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:04.917788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.919345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.934727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 338/661] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:04.936246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:04.952512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:04.953147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:04.953492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:04.953754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:04.961687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:04.963162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:04.967993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:04.976650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:04.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 324/662] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:04.978661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.016060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.048424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51f2ea-45c3-4a69-9483-862c316b810e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.084450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9543489-59c7-4683-8ef5-d8ffa897574a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.086318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.086929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.087234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.087595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.088239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.088540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.089906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.102097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 339/663] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.102984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.119284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.119887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.120162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.120426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.121054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.121334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.122683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.134543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 325/664] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.135374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.193539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.209354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d10e-e4da-4966-a95a-328856f9eea9 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.233766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.240331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef603-d616-4b4b-9b40-5c60a2b00d27 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.247447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.248089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.248440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.248867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.260914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:05.269476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:05.270503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:05.271615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 340/665] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:05.272778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.284085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.284971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.285390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.285803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.286927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.287431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.289690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.298529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 326/666] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.299409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.423624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cbc15-9158-4957-a568-553c2cd538de None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.425967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f037447-0d9e-40ca-acb6-67b35457a4ad None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.462785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.463385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.463660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.463938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.470114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.471403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.471403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.471403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.471686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.472458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.472846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.472951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.474414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:05.484493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:05.485426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:05.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 327/667] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:05.486887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.497878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 341/668] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.498771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.504761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.519575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.602120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7baf26-928e-4f41-acee-5b673e962987 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.615057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddefa6-1273-44e2-8345-5647f15cdabb None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.638595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.639977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.640274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.640561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.647748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.649217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.652883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.653721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.654185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.654800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.655736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.656269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.657881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:05.661389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:05.662315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 328/669] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:05.663270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.671719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 342/670] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.672812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.747716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.757613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.780388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a6b8-15f8-43ff-bc65-69dbcd11b6e4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.787263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663a10-38d1-4256-886b-271cf01d89b0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.815880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.816536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.816853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.817128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.817816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.818119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.819610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.826252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.827256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.827539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.827813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.831966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 329/671] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:05.832743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:05.911081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.913052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.939249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176aee8-57bf-4abe-a620-6ac1eddefca8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:05.977944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:05.978550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:05.978828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:05.979131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:05.979817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:05.980117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:05.981540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:05.987409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:05.991189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.000402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 330/672] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:06.000402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.039936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 343/673] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:06.040805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.106505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b959-48f9-47e6-ae7e-23a985d59cb3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.143952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.144585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.144862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.145130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.149689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8eb-e440-487a-975f-4243d7524d75 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.156558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.158016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:06.173455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:06.174451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:06.175361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:06.175361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:06.175361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:06.179368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 331/674] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:06.180215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.194003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.194757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.195059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.195330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.196053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=87af1d19163043ffaa3412487446b02f) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:06.196353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.197788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.215034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 344/675] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/role_assignments?user.id=87af1d19163043ffaa3412487446b02f&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:06.215034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.257565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.258110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.313251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb7a75-964b-49e6-9279-4b1feb0f25c5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.350712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.351517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.351888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.352256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.353088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=97d15d2823be4939a617603cfdad0aef) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:06.353495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.355072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.359964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9fd89-2a88-412c-93df-eda21ae6af6c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.362108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 332/676] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/role_assignments?user.id=97d15d2823be4939a617603cfdad0aef&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:06.363150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.415953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.416665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.417032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.417393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.432035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.433756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:06.453886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:06.455162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:06.456104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:06.456104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:06.456104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:06.456629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 345/677] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:06.463044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.499948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236ba9d-2b81-4180-bea4-418278085469 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.518892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.539584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.540674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.541152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.541654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.542505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:06.543036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.544875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.557750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.559979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 333/678] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:06.565966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.581745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd964e01-09e1-4c6e-a88f-9487db9c2833 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.621057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.621693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.621934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.622191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.622839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:06.623520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.624771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.637934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 346/679] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:06.643513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.687697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d53eb3-5abd-47a0-a208-1e3161f080fa None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.730464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.731201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.731474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.731807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.740285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.741431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:06.756690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:06.757710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:06.758895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 334/680] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:06.764833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.794829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271e9eb-15be-41e6-9b3a-64e8f971dcec None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.805844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.837556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.838625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.839195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.839778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.849593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.851512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:06.865033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:06.865974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:06.866884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:06.869851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 347/681] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:06.876929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.877990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:06.897033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbbc9f-fd9c-4960-8277-5daca2d68fa8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:06.933895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:06.934935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:06.935541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:06.936076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:06.937481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:06.937995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:06.939747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:06.959195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 335/682] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:06.975720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:06.992202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954662-571d-4388-b5d5-99dd2de2e4e0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:07.027532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:07.028154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.028430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.028700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.029557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:07.029922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:07.031401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:07.047489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 348/683] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:07.055078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:07.152406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:07.166289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:07.178858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c336047-269f-4ca9-a7f9-974c88e87f92 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:07.193691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b58a2-fb8c-45fa-b190-9c31544aa15f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:07.228199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:07.228807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.229082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.229349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.230078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=3acf83ba003e4e02a5ddd87c2714792d) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:07.230429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:07.232072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883e7c8-7175-4462-a805-4d9b6371bdcd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:07.239214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 349/684] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:07 2022] GET /identity/v3/role_assignments?user.id=3acf83ba003e4e02a5ddd87c2714792d&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:07.250945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:07.251728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.252077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.252420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.331148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:07.332740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:07.374807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e575-4edb-472d-bbd7-2554ca4826e4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.375738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e575-4edb-472d-bbd7-2554ca4826e4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.376001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e575-4edb-472d-bbd7-2554ca4826e4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.377892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 350/685] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:07.383956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.384565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.385030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.405549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:07.407159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:07.473405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:07.483548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 336/686] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:07.489700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:07.512723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 351/687] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:43:07.602988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b36d-8a05-4b6a-a84b-6dd84a2ab846 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:07.639680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:07.640368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.640639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.640987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.652048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:07.653548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:07.669360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:07.670398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:07.671464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:07.671464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:07.671464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:07.671464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 337/688] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:07.677027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:07.745276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:07.796042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:07.818735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb526c-4566-4def-a73e-65ad7edecc1f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:07.859547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:07.903100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:07.903728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:07.904005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:07.904274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:07.904998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:07.905293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:07.906735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:07.933102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 352/689] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:07.939117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:07.947016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08a541-f396-44ab-85dd-a19de423f563 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.016932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.017965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.018438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.018917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.050004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:08.053361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c105bb-6642-4efd-8a31-5f3f18401821 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:08.055982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:08.056906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:08.057761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:08.060432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 338/690] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:08.070931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:08.072158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.095415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.096377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.096986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.097527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.107499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.110073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:08.124210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:08.125151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:08.126017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:08.128916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 353/691] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:08.134650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.244029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ce73-1a36-4a80-895e-3c44f58c647f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.284151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.285200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.285775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.286219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.287271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:08.287764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.289763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.302802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5e7c0-39ab-48f1-a3c4-4b341bc28a3c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.305001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 354/692] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:08.312036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.319041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:08.365126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:08.372221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.372848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.373228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.373509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.374181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:08.374478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.376488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.386098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 339/693] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:08.398053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.510969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c52ff-f114-4957-a38a-ae7d746e9126 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.546020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.546825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.547243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.547598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.558717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.560369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.560956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541c8ce-5708-4f1b-82d8-4d2bbb42b34c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:08.575565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:08.576781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:08.577889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:08.577889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:08.577889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:08.583034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 340/694] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:08.585116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.631486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:08.655765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.656399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.656683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.656955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.657701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b98ce88b48c34731a1eae85d23de837a) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:08.658006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.659504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.675507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 355/695] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/role_assignments?user.id=b98ce88b48c34731a1eae85d23de837a&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:08.707778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e80c-7204-4b68-943a-dff95d2e4711 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.745704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.746331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.746611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.746881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.747589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:08.747891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:08.749462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1abfb3-e6e8-448e-863d-221164f5a497 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:08.766828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 341/696] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:08.772283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:08.780316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56327cbc-791e-4137-b8fd-ce13f320c4fe None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.780316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56327cbc-791e-4137-b8fd-ce13f320c4fe None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.780811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56327cbc-791e-4137-b8fd-ce13f320c4fe None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.782518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 342/697] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:08.799611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.800050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.800380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:08.825816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:08.835179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:08.911410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 343/698] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:43:08.946505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:08.949765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2298-9bd7-4522-aabd-b3cbf6948173 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:08.991430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:08.992113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:08.992439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:08.992712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.000314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.001773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e2746a7b-7481-478d-89c5-007e24b3aab3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:09.013838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:09.014778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:09.015736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 356/699] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:09.021083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.035402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.116374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244455-8c4c-443b-ac9f-499def0342cc None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.116838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244455-8c4c-443b-ac9f-499def0342cc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.117475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244455-8c4c-443b-ac9f-499def0342cc None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.123227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 357/700] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:09.125277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.125665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.126037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.132125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2e630-f579-4920-82a8-a1d7f16141a7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.145169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:09.146592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:09.169346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.169878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.170251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.170618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.171381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:09.171680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.173105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.200552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 344/701] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:09.205996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.234260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 358/702] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:43:09.235160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.289914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.328792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b864f-e150-49ff-9dec-e3750a92905a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.352859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.366604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee92b7-6e67-499b-ad5d-22fea2ad2894 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.370353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.370969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.371272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.371546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.422660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.423263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.423586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.423856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.424457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.425890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:09.438975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:09.441300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:09.443200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 359/703] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:09.448667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.502794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.504294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.564958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2361e-2678-4574-bbb6-541664186a33 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.627863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.629837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.630456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.630714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.630979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.631649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:09.631947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.633363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.645432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 360/704] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:09.652732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.677554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.678549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 345/705] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:43:09.679624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.795058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db1aef-5ffe-478c-a56f-29d8c04c6d1a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.796156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db0df-6ed8-4c1c-9bd3-3b45a952e78f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:09.834029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.834591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:09.834715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.834905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.835225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.835325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.835534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.835761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.836568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:09.836892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.838951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.845805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 361/706] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:09.847281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.848340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:09.850156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-70e08cfd-752c-4ed5-8262-43f2fb779793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:09.867353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:09.868694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:09.869865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:09.869865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:09.869865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:09.873667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 346/707] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:09.875171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e918b5-eed3-4229-8a7a-d5a88465b5dc None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:09.875460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e918b5-eed3-4229-8a7a-d5a88465b5dc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:09.875739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e918b5-eed3-4229-8a7a-d5a88465b5dc None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:09.877406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 347/708] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:09.878548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e918b5-eed3-4229-8a7a-d5a88465b5dc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:09.903640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.965768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:09.988047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309a2ed-a1ce-4ba5-b358-54ca40f6865b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.015810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e918b5-eed3-4229-8a7a-d5a88465b5dc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.032415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.033060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.033340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.033614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.045501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.047382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-58cf3feb-6a58-4a0b-a203-e4b9b85c031f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:10.063358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:10.064633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:10.065762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:10.065762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:10.065762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:10.071110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.071263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 362/709] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:10.071263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.071263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.071263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.071677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.071805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.071988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.072744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:10.073043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.074578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.089616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:10.090837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:10.095089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 348/710] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:10.095990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.181813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 363/711] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:10.182815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.216268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.259740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.261153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d0f94-e6b0-410c-8b1a-d0884fd6299e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.321348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aa289-f0ce-496d-a049-3b5a949829ef None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.330371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.330998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.331300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.331574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.332223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:10.332524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.333926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.366191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 349/712] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:10.372596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.401883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.403186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.404434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.405494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.406230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.411944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.435434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 364/713] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:43:10 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:10.478425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.617044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300a06-e6d7-44da-a265-81bb69073099 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.617394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1b9-973f-4b1c-8c39-48a7cff39c8c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.659042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.661617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.662261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.663911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.664200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] PATH_INFO: `/v3/services/network` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.664780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.666245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:10.679273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:10.680268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:10.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 365/714] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:10.686433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.687038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.687787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.688338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.688753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.698126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.700326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.704183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:10.718696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:10.719877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:10.720742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:10.723550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 350/715] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:10.729317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.804885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3455e-e2cd-47ca-b114-5546f983f127 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.834173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f648d7-ba4e-4701-acd0-3fc405fdb11d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:10.841105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.842109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.842566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.842934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.844089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] RBAC: Adding query filter params (name=network) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:10.844972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.845990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.851432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 366/716] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:10.856833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.875243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:10.876148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:10.876600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:10.877005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:10.877862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:10.878352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:10.880085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:10.898665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 351/717] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:10.899825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:10.916052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.966442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:10.976692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dc7e9-f3ec-4d84-94c1-7d07eacb29de None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:11.013184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0f380-739f-4f6b-a958-43f22f53ff23 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:11.014918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:11.015765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:11.016142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:11.016513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:11.017594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] RBAC: Adding query filter params (type=network) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:11.017594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:11.019174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80eabe-a3f3-465d-aea7-d2ee5eae5ad1 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:11.024476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 367/718] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:11.054179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:11.055111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:11.055628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:11.056037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:11.056948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:11.057460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:11.059093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7326daa-eb76-4c66-b3c2-b89d2e75e80d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:11.066369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 352/719] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:10 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:11.334647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcd51-4b57-4bce-933c-0565892317cb None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:11.335099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcd51-4b57-4bce-933c-0565892317cb None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:11.335506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcd51-4b57-4bce-933c-0565892317cb None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:11.337287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 368/720] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:11.342713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:11.343648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:11.344118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:11.370023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:11.371589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:11.452390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 353/721] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:11.739440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfcd51-4b57-4bce-933c-0565892317cb None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:11.883826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfcd51-4b57-4bce-933c-0565892317cb None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:11.905073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:11.942911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:11.943583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:11.943861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:11.944134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:11.965074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:11.970281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:11.971268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:11.972115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:11.976443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 369/722] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:11.980114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.066835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:12.116642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628867bd-433f-44b3-869f-bbcbacf913cd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:12.158367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:12.158972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.159271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.159543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.160311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:12.160709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:12.162072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82740324-4911-4002-9e3d-5961482f8f21 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:12.170828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 354/723] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:12.176933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.188667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59acb0-605d-489c-9886-d95eba72b501 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.189285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59acb0-605d-489c-9886-d95eba72b501 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.189768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59acb0-605d-489c-9886-d95eba72b501 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.191583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 355/724] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:12.196759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.197282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.197780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.229711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:12.230897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:12.353033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874628-3e64-4e5b-8d94-1837826ac313 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:12.365026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 356/725] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:12.399564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:12.400216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.400490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.400761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.414132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:12.414132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:12.435109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:12.436146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:12.437039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:12.437039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:12.437039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:12.441126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 370/726] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:12.442452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.505991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:12.575868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a80535-b24d-4801-9100-57e16918c0ce None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:12.620297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:12.620825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.621100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.621370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.622002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:12.622306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:12.623885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:12.639832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 357/727] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:12.645536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.666186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.695560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:12.844400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc72a0-e55c-4b99-9088-e616606aac9e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:12.860608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cbbbd-b374-43fe-867e-9f8cd0781f09 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:12.870831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:12.912501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:12.913213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.913394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.913668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.928990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:12.929620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:12.929898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:12.930170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:12.930849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:12.932252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:12.936965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:12.938460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:12.959052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:12.960025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:12.960949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 371/728] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:12.962488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:12.966837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 358/729] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:12 2022] POST /identity/v3/services => generated 251 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:13.061528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:13.081662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be391911-7b86-4505-9228-cbb68456a476 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:13.127375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:13.127916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.128191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.128460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.129202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:13.129502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:13.130939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc984fd-23d5-43b9-9772-2a27955663d2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:13.151029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 372/730] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:43:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:13.160272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:13.290542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d9a84-fa0b-4a37-9afb-1efa6d7f0e4e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:13.327247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:13.329337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c21c0-c39a-43e3-b936-cc31207133de None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.329619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c21c0-c39a-43e3-b936-cc31207133de None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.329886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c21c0-c39a-43e3-b936-cc31207133de None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.331171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.331514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 373/731] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:13.331929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.332396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.336546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19370b-5440-4801-883a-2e6b5265b227 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.336834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19370b-5440-4801-883a-2e6b5265b227 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.337109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19370b-5440-4801-883a-2e6b5265b227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.358523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ea19370b-5440-4801-883a-2e6b5265b227 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:13.359802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ea19370b-5440-4801-883a-2e6b5265b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:13.373515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:13.424184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:13.425651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:13.437619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 374/732] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:13.439511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e187-ab84-4c0a-b6d5-15579f85ebdc None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.439511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e187-ab84-4c0a-b6d5-15579f85ebdc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.439511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74e187-ab84-4c0a-b6d5-15579f85ebdc None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.441051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 375/733] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:13.446321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8d31e-4d64-443d-9e6c-560a1df1cc82 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.446625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8d31e-4d64-443d-9e6c-560a1df1cc82 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.446872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8d31e-4d64-443d-9e6c-560a1df1cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.467360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-daf8d31e-4d64-443d-9e6c-560a1df1cc82 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:13.468959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-daf8d31e-4d64-443d-9e6c-560a1df1cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:13.537681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:13.538008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 359/734] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:43:13.539810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:13.544270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 376/735] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:43:13.607441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:13.629279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5f2e6-8bbf-4450-8d65-05bc606d36e2 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.630199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5f2e6-8bbf-4450-8d65-05bc606d36e2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.630557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5f2e6-8bbf-4450-8d65-05bc606d36e2 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.632298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 377/736] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:13.637610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.637963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.638392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.643581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a69692-3dc1-4e0b-a433-9976f5b5325f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:13.657740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:13.659172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:13.686175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:13.687155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.687780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.688240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.708571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:13.710053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:13.725539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:13.726542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:13.727968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:13.727968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:13.727968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:13.727968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 360/737] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:13.732553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:13.815156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 378/738] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:43:13.829143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:13.839031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee4620-2734-45ec-a2ae-9677724d3425 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:13.874058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:13.874682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:13.874960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:13.875288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:13.875912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:13.876213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:13.877892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:13.880298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:13.895294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 361/739] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:13.896672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.001781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3850-b62a-4ac5-b59e-93cc55e11fc5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.008770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f511890-5b8f-41d9-aee6-d6dfc00e791f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.037983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.038620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.038892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.039440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.040358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.048401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.049252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.049628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.049921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.050063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.051768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.061819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:14.064507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:14.065486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:14.066376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:14.067881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:14.068759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:14.069574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 379/740] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:14.070567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.075750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 362/741] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:14.077419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.121979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.182643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76b6dd-b2f9-442e-8ecf-8f3b9bb82d8a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.190368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031758d-3cf7-4ca1-af2b-7db06db95961 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.219801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.220664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.221122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.221387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.228322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.228926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.229200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.229615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.230154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:14.230453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.231954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.232372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:14.237752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:14.238701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 380/742] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:14.239659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.242977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 363/743] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:14.243933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.304729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.363095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bf0a4-bce4-4157-b418-e78218d29241 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.403380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.404001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.404396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.404674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.405387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:14.405692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.407615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.432902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 381/744] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:14.432902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.480754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.484527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc37569-d298-434b-acd9-d0210e57c83e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.524298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.525193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.525474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.525748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.526576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.528280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.543754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484af22a-8e19-4335-9868-7c14e14a7eae None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.551061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 364/745] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:14 2022] POST /identity/v3/services => generated 240 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:14.551061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.555260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.602167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.603688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.604082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.604486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.605336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:14.605805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.607486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.620469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 382/746] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:14.621571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.679565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a53b3-0337-4f7d-bc56-e6cd4ee05312 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.722973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.723698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.723972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.724242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.735235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.737258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.749912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691646-1ffc-4be6-8965-e456d7b36522 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:14.755501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:14.756440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:14.757343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:14.757343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:14.757343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:14.757343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 365/747] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:14.757343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.774581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.794024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:14.794810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:14.795219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:14.795592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:14.796401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b98ce88b48c34731a1eae85d23de837a) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:14.796768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:14.798875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:14.806668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 383/748] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/role_assignments?user.id=b98ce88b48c34731a1eae85d23de837a&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:14.806668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:14.877867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:14.973097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b6e5f-22db-4dd5-a244-f7efd3a1f776 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:14.975324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f9340-5383-4e41-93b2-056678a5cc67 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.017031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.018542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.019162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.019663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.020614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:15.021170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.022713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.023151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.023417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.023687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.023964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.039871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 384/749] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:15.044165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.045624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.046336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:15.074409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:15.075431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:15.077794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:15.077794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:15.077794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:15.082652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 366/750] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:15.083731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.179479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.188263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b595e7-ca2a-4988-ac76-a9e74e04a42f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.231699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.233615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.234118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.234659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.235366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.244583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.246426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:15.261663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:15.262709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:15.264335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:15.267537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 385/751] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:15.272771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.326894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9045-bf82-4163-994a-ffdba0c143da None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.400411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cbe1c-2c80-4114-9d80-62e9843606fe None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.400627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.401166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.401447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.401719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.402601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:15.403138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.405851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd62a-daba-4423-b663-303c7aafdeaa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.424289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 367/752] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:15.426166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e380d-07cd-40be-80bb-1f63ff23d1f5 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.426461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e380d-07cd-40be-80bb-1f63ff23d1f5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.426732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e380d-07cd-40be-80bb-1f63ff23d1f5 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.431379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 368/753] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:15.432540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e380d-07cd-40be-80bb-1f63ff23d1f5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.455741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.456366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.456646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.456921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.457667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:15.457973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.459923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84139f-2716-480b-81e8-2d92efe7d0ff None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.482685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 386/754] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:15.484749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.485264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.485737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.507329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:15.508959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:15.543419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.564148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e380d-07cd-40be-80bb-1f63ff23d1f5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.591533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 387/755] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:15.592850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.594123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.612531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.612531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.612531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.612531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] PATH_INFO: `/v3/users/nova` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.626320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.627833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:15.655096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:15.656142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:15.657108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:15.657735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 369/756] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:15.661819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.773383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058e680-51c1-46c9-844c-768d8e494bea None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.809647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.810296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.810646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.811169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.812086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:15.812403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:15.813844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:15.833325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 370/757] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:15.838835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:15.863353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8bbea-c4ed-4316-83ad-f815dbc665ef None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.895951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.929600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:15.930685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:15.931189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:15.931694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:15.948627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:15.959273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6ddf-7b63-4644-9a9d-01e4679bb488 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:15.999886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.000927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.001480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.001991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.014624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.016720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.022092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.023710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:16.035519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:16.036648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:16.037557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:16.037557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:16.037557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.039528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 371/758] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:16.040747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.132222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 388/759] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:16.133218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.151620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.161926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae75deb-4d67-4db0-8e7f-ecbd5a568f12 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.179440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.206047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.207187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.207612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.208147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.208927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.210992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:16.225003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:16.226274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:16.227272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.230638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 372/760] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:16.231720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.251163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c34094-4f3a-47c3-a956-9d14b665d19f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.287760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.288479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.288853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.289575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.289979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.290342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.291832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.305212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 389/761] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.306136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.351044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509c6e1-50ad-4253-9d51-34782320619e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.391104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.392639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.393172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.393753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.410840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.415357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.417695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.421707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f5ff6-32f6-430f-b342-2ede4dd2f0ee None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:16.436463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:16.437527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:16.438442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:16.438442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:16.438442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.440552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 373/762] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:16.442099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.456803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.460059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.461826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.462252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.462679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.463521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.463965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.465726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.476440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 390/763] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.477553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.562740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7d90f-a259-46bb-b410-210d5eb925fb None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.581787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5693-a07c-454d-a805-42bf814d8683 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.605138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.605930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.606320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.606712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.616009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.617705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.621246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.622047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.622468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.623047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.624024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.624484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.626008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:16.631728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:16.632825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:16.633995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.635716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 374/764] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:16.637193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.642719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 391/765] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.643818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.668698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.721579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.750112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ffd7-fd3f-41d8-8efc-305cdbbd0574 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.755580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73944-2d14-4be6-8c36-c622ed3f4423 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.790500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.791267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.791613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.791992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.792710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.793084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.794641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193c22d-28e0-4f6a-8b0b-82be1985c280 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.795765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.796439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.796779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.797044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.797671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.797969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.799340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.810543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 375/766] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.810543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.813193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 392/767] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.814601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43920a6-cd69-4046-8684-62f9016f0673 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.814986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43920a6-cd69-4046-8684-62f9016f0673 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.815413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43920a6-cd69-4046-8684-62f9016f0673 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.819068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 393/768] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:16.819068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43920a6-cd69-4046-8684-62f9016f0673 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.906680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.924797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140d58d-e2ab-4ce4-a7e0-3e6184cb5b0b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.928913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43920a6-cd69-4046-8684-62f9016f0673 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:16.963090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.963415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:16.963916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.964163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.964335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.964557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.964740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.964939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:16.965719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] RBAC: Adding query filter params (interface=public, service_id=847e06eaf29041818735e7fda540ad49, region_id=RegionOne) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:16.966086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.967736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.968745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:16.974179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:16.976254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 394/769] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/endpoints?service_id=847e06eaf29041818735e7fda540ad49&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:16.976780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:16.979879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a5727186-e640-4f43-b1db-d135af447aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:16.993605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:16.994758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:16.995686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 376/770] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:16.996168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:16.996235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:16.996510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.016120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:17.017347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:17.088609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 377/771] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:17.089496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.090373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1717ab-59ae-4a78-bf9d-35619caa00f7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.128403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.129187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.129530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.129865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.131895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=28f8f65dc678403081189e31ef5f8a10, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.133412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.148391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 395/772] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:16 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:43:17.149429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.167374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:17.233335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1908631-221a-4fe0-a0f0-a2dd688fd78b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.271182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:17.288827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.289462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.289748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.290284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.290921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.292329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1a179-6ae9-440f-89b6-b7b630905486 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.292749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9a01-eb85-4394-b631-f945b77988b2 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.299889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 378/773] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:17.301779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091674-a2cd-4f42-bac2-109a4ba3d173 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.301779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091674-a2cd-4f42-bac2-109a4ba3d173 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.301779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091674-a2cd-4f42-bac2-109a4ba3d173 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.304847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 379/774] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:17.312460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.312773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.313393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.338352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:17.339611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:17.342503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.343792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.344230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.345230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.346130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:17.346575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.348314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.365238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 396/775] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:17.371848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.437168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 380/776] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:17.438922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.465134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:17.516073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0beafd3-a2c9-4723-8c61-d87424880b83 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.561829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.563260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.563822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.564332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.565436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=9aabe3099e2147e29be189622d916500) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:17.566061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.568383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.575568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 397/777] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/role_assignments?user.id=9aabe3099e2147e29be189622d916500&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:17.579063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaf942-30d1-48fa-9eca-f898382ef238 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.597578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:17.617807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.618624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.619017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.619411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] PATH_INFO: `/v3/services/network` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.620246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.621724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:17.631800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:17.633044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:17.634068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:17.635892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 381/778] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:17.641818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.772725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.792266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449a3-38da-4aa3-aea3-cb1e225c7557 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.827772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:17.835442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.836110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.836384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.836697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.837337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] RBAC: Adding query filter params (name=network) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:17.837633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:17.839029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:17.845902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 398/779] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:17.852483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:17.910137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776737e6-fbd8-41e0-85a2-e312ef40eb07 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:17.950872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:17.952551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:17.953032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:17.953476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:17.965478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:17.972668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:17.973831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:17.974889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:17.982611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 382/780] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:17.984448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.047730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:18.081086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb512e21-93ab-4e01-9454-877bcd0361cc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.158070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.159707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.160097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.160470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.161232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] RBAC: Adding query filter params (type=network) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:18.161633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.163463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.175032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2b2e-20a0-4042-a7a0-503dad9e4048 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.175224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 399/781] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:18.175458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.245996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.246646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.246949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.247251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.247983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:18.248483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.250002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.281692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 383/782] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:18.283300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.303747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:18.394343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6255-4309-47e0-a4a2-75930a6b0103 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.465825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.466516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.466901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.467373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.468556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] RBAC: Adding query filter params (interface=public, service_id=ec29ea6381c644adb1ccad0c768bc8b5, region_id=RegionOne) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:18.468864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.473312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.479875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fee90-8e0b-4936-8412-229ead5e0fa0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.486657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 400/783] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:18 2022] GET /identity/v3/endpoints?service_id=ec29ea6381c644adb1ccad0c768bc8b5&interface=public®ion_id=RegionOne => generated 194 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:18.487885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:18.491792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.547570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.548205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.549061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.549332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.569791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.571191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:18.596914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:18.597930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:18.598805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:18.598805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:18.598805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:18.601357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 384/784] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:18.607920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.681210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f311f0-2e24-41fb-a6b7-06c7ab34e7cd None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.729964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:18.742840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571808-4325-4584-b23f-93549f771c7c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:18.760528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.761471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.761919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.762357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.763168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.765212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.779230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 401/785] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:18 2022] GET /identity/v3/services => generated 772 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:18.801069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:18.802071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:18.802586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:18.803142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:18.804039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:18.804511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:18.806054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:18.821022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 385/786] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:18.828143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:18.963973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b6d4c-8e9f-4c11-9236-48e9d7d6d47a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:19.011575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:19.012217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.012498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.012777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.022069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:19.023781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:19.037160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:19.038164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:19.039140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:19.043243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 402/787] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:19.047027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:19.076354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:19.181461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd83888-ed53-4168-be00-d8b6b5979605 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:19.224614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:19.225290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.225575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.225849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.226630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:19.226968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:19.228609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:19.252559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 386/788] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:43:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:19.255261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:19.308338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:19.368051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225ac24-4ca9-425c-8908-fc72ed2d8227 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:19.406785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:19.407545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.407878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.408173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.420135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:19.421833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-dba0ed61-7c71-4617-81e6-ce58315354d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:19.437532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:19.438523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:19.439447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:19.439447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:19.439447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:19.443111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 403/789] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:19.445113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:19.557986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:19.595209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32ba94-cf1f-474e-8548-55e3c03d09d5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:19.637916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:19.638486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.638765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.639063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.639714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:19.640015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:19.641391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:19.644825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723791e1-791d-4d99-be10-d6546e21eb8e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.645116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723791e1-791d-4d99-be10-d6546e21eb8e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.645393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723791e1-791d-4d99-be10-d6546e21eb8e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.647195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 404/790] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:19.652170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc4fe-01ac-4cd9-b9da-1864dd254211 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.652454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc4fe-01ac-4cd9-b9da-1864dd254211 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.652728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc4fe-01ac-4cd9-b9da-1864dd254211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.665577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 387/791] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:19.675289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:19.686552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e06cc4fe-01ac-4cd9-b9da-1864dd254211 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:19.687790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e06cc4fe-01ac-4cd9-b9da-1864dd254211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:19.782814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352625-a50f-46d5-898a-2ffaf0e5a4d3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:19.797527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 405/792] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:19.799164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff285-7b1e-47f0-9e58-1f806b0615e9 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.799628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff285-7b1e-47f0-9e58-1f806b0615e9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.800119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff285-7b1e-47f0-9e58-1f806b0615e9 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.801956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 406/793] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:19.811389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4c07-09e0-4b08-8eb3-f621f821e0d5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.811803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4c07-09e0-4b08-8eb3-f621f821e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.812200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4c07-09e0-4b08-8eb3-f621f821e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.821441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:19.822104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.822386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.822684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.827426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:19.836485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:19.837999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:19.862990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:19.863981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:19.864882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:19.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-544a4c07-09e0-4b08-8eb3-f621f821e0d5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:19.871025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-544a4c07-09e0-4b08-8eb3-f621f821e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:19.874581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 388/794] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:19.875491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:19.913280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:19.984490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 407/795] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:19.986085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4672-f22c-4e52-81e5-011c8b877411 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.986374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4672-f22c-4e52-81e5-011c8b877411 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.986644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4672-f22c-4e52-81e5-011c8b877411 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:19.991994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 408/796] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:19.996471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:19.996756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:19.997027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.033516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689042b8-d6dc-46a7-8dfd-ee450e00f75d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.048050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:20.049270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:20.071350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.072032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.072312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.072588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.073340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:20.073649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.075111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.088858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 389/797] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:20.095860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.163022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.165358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 409/798] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:20.166174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.238354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b7f2b-b682-44b3-9326-62379ecd956e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.262277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.270909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d6a00-218b-408c-8965-f6ef807f6d9b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.306105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.306749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.306879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.307104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.307323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.307516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.307824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.307926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.308137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/b98ce88b48c34731a1eae85d23de837a/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.309281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.310262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=b98ce88b48c34731a1eae85d23de837a, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.312586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:20.318574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:20.319683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:20.320863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 410/799] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:20.321845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.329713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 390/800] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:20 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/b98ce88b48c34731a1eae85d23de837a/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:43:20.330793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.436272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efac-f4e9-4440-8f12-e5d70dc7b2b7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.473988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.474641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.474918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.475226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.487231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:20.492664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:20.494631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:20.495996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 411/801] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:20.503791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.529922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.567198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251fa3-e2ea-473a-a78b-4c8ce24452fc None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.584145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.612941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.615076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.615504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.615926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.616697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:20.617140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.618679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.628907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 391/802] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:20.630090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.643945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2a901-c587-4f5f-8c02-6bab9a2e13b1 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.681535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.682380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.682780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.683206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.684024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:20.684428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.686756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.699983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 412/803] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:20.699983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.763908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fc60-da4e-4ac6-b2a2-09afc6e85005 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.784573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.809951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.811380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.812116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.812552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.814272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de958d81-7b1c-4e30-832d-1a65c201a149 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:20.825829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:20.833380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:20.834516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:20.835591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:20.843574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 392/804] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:20.843574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.866839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:20.867910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:20.868387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:20.868854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:20.870325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:20.871131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:20.873103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:20.887096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 413/805] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:20.889729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:20.895643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:20.980840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed865-2827-4061-987f-f62b12cc01e7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.004030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172fd7-a214-40c0-8ba2-5abd58d4dbe5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.029460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.030631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.031491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.031919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.042682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.043486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.043881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.044621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.045656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:21.046067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.047223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.047702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.049034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.054238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.064151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 414/806] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:21.065139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:21.068382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:21.069518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:21.070480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:21.070480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:21.070480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:21.077005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 393/807] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:21.077005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.177816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655cc5-75f9-46cd-8fdf-4c12447b6a46 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.182963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.194798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a840d-31d4-4185-a0f5-f98daa7a28a3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.213866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.214480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.214758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.215054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.215723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.217198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.230596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.231330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.231552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.232119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.240921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 415/808] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:21.240921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.254581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.256236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-1475cdec-ce59-4a3c-b814-b62bf159591e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:21.275443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:21.276532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:21.277588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:21.277588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:21.277588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:21.277588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 394/809] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:21.278761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c649-f1f5-49ef-aecb-07e76b357744 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.279387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c649-f1f5-49ef-aecb-07e76b357744 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.283262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c649-f1f5-49ef-aecb-07e76b357744 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.285417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 395/810] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:21.286915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.287353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.287722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.308011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:21.309557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:21.326561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.396798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff103b7-3c2a-4c21-92c9-52f5089440e9 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.419434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 396/811] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:21.420368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.479749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.480388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.480670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.480947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.481655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:21.482061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.483599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.502031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 416/812] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:21.503127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.504123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.533448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ff934-3d93-4ee2-a860-3acb47834463 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.572781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.573422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.573722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.574022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.574670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:21.574975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.576426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.588830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 397/813] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:21.595778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.641509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e1956-2a70-4799-9d3f-2997fb0b4779 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.667282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.710083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.710733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.711032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.711295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.729493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.730955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:21.757102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:21.758069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:21.758992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 417/814] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:21.763927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.768174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53b02-7293-4f28-82b4-1c738b45e464 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.798849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:21.805259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.806034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.806311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.806583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.814989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.816770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:21.829245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:21.830241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:21.831368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 398/815] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:21.831991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:21.901871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3850984-d72b-491a-b961-5c7feb0ff195 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.956433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:21.957438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:21.957715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:21.957982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:21.958980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:21.959308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:21.960760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:21.961143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f351a-5508-4432-bf54-143814b0b7c0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:21.984784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 418/816] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:21.985733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.005963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.006613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.006882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.007268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.007917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.009591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.020694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:22.021158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:22.022155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:43:22.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:22.027176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 399/817] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:22.028043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.088400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.119815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734409a-978f-4381-a355-9e05d1adb45f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.156414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.156810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb43cc-7745-42ad-8308-97ea5dade5a7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.157056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.157330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.157600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.158321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:22.158685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.160298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.181746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 419/818] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:22.182588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.203562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.204401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.204675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.204990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.280171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.281668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.292681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.308710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba21648-9dbc-4a78-b173-5eba36ea9840 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.343917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.349092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.349713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.349989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.350262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.350897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] RBAC: Adding query filter params (name=network) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:22.351230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.352602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.357761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 420/819] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:22.358629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.401415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 400/820] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:22.402508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.467320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614acbe-d9fd-4d15-9d55-c75c1389338d None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.507944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.508579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.508852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.509114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.516497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc78ff-6e75-476d-8573-095419daed31 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.520108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.521561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:22.536959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:22.538063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:22.539457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:22.539457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:22.539457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:22.539457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 421/821] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:22.539457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.546183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.560320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.561005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.561344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.561617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.562282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] RBAC: Adding query filter params (type=network) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:22.562585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.564054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac52959-6f30-42aa-95d1-a82790849fc6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.569341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 401/822] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:22.570516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af3e5-ba69-4c13-8681-d2a7f4f60510 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.570800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af3e5-ba69-4c13-8681-d2a7f4f60510 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.571102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af3e5-ba69-4c13-8681-d2a7f4f60510 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.576756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 402/823] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:22.576756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af3e5-ba69-4c13-8681-d2a7f4f60510 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.602063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.660670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812d258-1a94-41c4-ab79-85eecdac1df6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.689835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af3e5-ba69-4c13-8681-d2a7f4f60510 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.698988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.699645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.699915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.700181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.714598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.716387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.731642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.732323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.732678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.732962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.733596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:22.733897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.735318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48bdc6-55b1-4f91-a8a3-fb35cd7c9381 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:22.736705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:22.737859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:22.739148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:22.739148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:22.739148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:22.739148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 422/824] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:22.739148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.755143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 403/825] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:22.756963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.757395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.757949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.776923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:22.778115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:22.800834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.849443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06ada-ac89-4d19-a96e-824fdd0a171a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:22.863195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 404/826] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:22.863195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.874982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:22.886629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:22.887269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:22.887546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:22.887818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:22.888555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:22.889961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:22.916580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 423/827] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:22.917533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:22.989053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244500d6-5cd0-4c74-853d-2cddb4ceb396 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.059968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.060203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcbb9e-20db-4988-9a4a-e4b0cf1069fa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.063250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.063250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.063250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.063250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:23.063250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.064117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.074556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.085306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 405/828] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:23.094903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.106384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.107097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.107385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.107658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.117688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.119521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:23.134972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:23.135985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:23.136901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:23.139115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 424/829] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:23.143050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.195770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.260011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88769e9-18cf-4a2f-be52-305dee891b0a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.272518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6357fa-e6c7-4ccf-9ed9-1c249e97aed7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.300840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.301490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.301786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.302073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.310351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.311867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.316000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.316621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.316903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.317180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.317888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:23.318196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.319979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:23.323641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:23.324498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:23.325373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 406/830] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:23.325970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.333550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 425/831] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:23.334279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.376837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.468860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ced73-3a84-4d22-9011-f61c567ea66b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.472564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d783054-de86-46f9-bfda-68ab3f0858ad None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.496513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.506210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.506501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.506852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.507145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.507261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.507422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.507511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.507696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.508388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:23.508684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.510112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.517713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:23.521725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 426/832] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:23.522535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-95dc2df5-e946-4cf5-92bb-10b40d7e5f23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:23.522917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:23.523877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:23.524731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:23.528795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 407/833] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:23.530128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209024f-ef16-475a-a8d0-ea0f6ba32229 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.530406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209024f-ef16-475a-a8d0-ea0f6ba32229 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.530675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209024f-ef16-475a-a8d0-ea0f6ba32229 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.532003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 408/834] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:23.532687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209024f-ef16-475a-a8d0-ea0f6ba32229 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.639517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209024f-ef16-475a-a8d0-ea0f6ba32229 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.678368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.678991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.679298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.679578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.680314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:23.680622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.682141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.689238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 409/835] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:23.690107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.733744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.751474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:23.759062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb94faa-14d2-4b9a-a015-c39dd2b148e8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.802496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.803259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.803630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.803924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/9aabe3099e2147e29be189622d916500/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.816657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfc33a-3f1a-4ac2-bc6e-2e6537fb9b2e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:23.820600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=9aabe3099e2147e29be189622d916500, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.822076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffe165-147e-4d04-ae16-0d34cbf24964 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.837759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 427/836] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:23 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/9aabe3099e2147e29be189622d916500/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:43:23.839133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.839418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.839690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.860410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:23.861894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:23.866336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:23.867368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:23.867841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:23.868302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:23.869211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:23.869729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:23.871429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:23.882170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 410/837] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:23.888556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:23.962177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 428/838] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:24.028945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6470e1-3f98-48e3-8484-e33a1b9c4583 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.047550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.066778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.067524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.067808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.068082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.080677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.082143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:24.097793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:24.098781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:24.100440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:24.100440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:24.100440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:24.100440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 411/839] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:24.106035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.177855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.220607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdc812-d043-409e-8cff-594c949728c2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.243491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.258125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.258980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.259423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.259884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.260764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:24.261229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.262741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.276014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 429/840] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:24.284195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.359269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c6bb9-9733-4c64-b0ef-b48b7f55bb62 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.400186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.400837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.401114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.401389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.401979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.405206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.418646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87d84-9d1d-4912-b98b-45b32e080efc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:24.421089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:24.422088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:24.422980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:24.430039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 412/841] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:24.431091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.450410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.462208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.463112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.463391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.463661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.468407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.476596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.478986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:24.495851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:24.497028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:43:24.498457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:24.502961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 430/842] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:24.503951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.556548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa73d-9941-4556-b747-fc2cd2d8a407 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.594294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.595720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.596182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.596897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.597728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:24.598364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.600707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.606368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 413/843] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:24.611600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.636434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278890-15c3-4cf2-a7f2-9cde7501be18 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.676763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.677692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.678657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.679174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.680383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:24.681115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.683474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.712450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 431/844] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:24.713563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:24.728261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.738762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f3367-b1d1-457f-b6cc-35271995d295 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.776076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.776748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.777042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.777293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.777924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:24.778226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.779898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:24.781378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:24.791048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 414/845] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:24.851960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c20f9f-2014-4909-8657-34c00a8fd81e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:24.890968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:24.891632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:24.892091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:24.892390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:24.970322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:24.972168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.021405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:25.078751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 432/846] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:25.084757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:25.195508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d869-0cae-4573-9f4d-37bf1d09f83d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:25.231477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:25.232468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.232927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.233372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.244781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:25.246493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:25.261626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:25.262630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:25.264139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:25.264139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:25.264139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:25.266015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 415/847] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:25.271862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:25.375051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353df47-7b69-4c8b-9e25-2bc02fbd4788 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:25.411076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:25.412000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.412459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.412879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.413674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:25.414120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:25.415682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.428464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 433/848] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:25.434125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:25.446255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:25.547147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b036fb3-0864-4a15-b9f6-b932e4546e9c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:25.582663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:25.583668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.584107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.584533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.591917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:25.593892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-34bc20d3-4df1-45bc-bbd1-160c924e6c02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:25.605717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:25.606929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:25.608164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:25.608600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 416/849] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:25.614555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:25.631960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:25.721453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348e36f-5d8e-4e63-9cff-3142fd290669 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:25.757658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:25.758561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.758986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.759428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.761141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0da277-e231-4a07-ab7f-1e42789f2b00 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.761603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0da277-e231-4a07-ab7f-1e42789f2b00 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.761704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0da277-e231-4a07-ab7f-1e42789f2b00 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.762606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:25.763339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 417/850] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:25.763524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:25.765145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.771915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb082f-a6c1-4d42-a679-ce1555cb6552 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.772194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb082f-a6c1-4d42-a679-ce1555cb6552 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.772470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb082f-a6c1-4d42-a679-ce1555cb6552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.783395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 434/851] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:25.789619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:25.798086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-63cb082f-a6c1-4d42-a679-ce1555cb6552 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:25.799296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-63cb082f-a6c1-4d42-a679-ce1555cb6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:25.862231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:25.882096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:25.887743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 418/852] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:25.889262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927942c-62d6-4ed5-b5fb-6c0adc4373bb None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.889543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927942c-62d6-4ed5-b5fb-6c0adc4373bb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.889816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927942c-62d6-4ed5-b5fb-6c0adc4373bb None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.891312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 419/853] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:25.896819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.897206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.897788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.906156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba509d-0773-4b2b-bf18-cabbb700f350 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:25.921015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:25.923199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:25.942199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:25.942962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:25.943400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:25.943744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:25.944579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b98ce88b48c34731a1eae85d23de837a) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:25.944931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:25.946433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:25.959302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 435/854] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:25 2022] GET /identity/v3/role_assignments?user.id=b98ce88b48c34731a1eae85d23de837a&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:26.012680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 420/855] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:26.194957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.293837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.300134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b88f-854e-42ba-833b-0a084d277eb7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.331912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:26.338790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.339791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.340285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.340795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.352544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:26.358443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:26.359429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:43:26.360469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:26.362818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 436/856] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:26.368760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.408106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ded66-40d9-4a97-a39b-7d5c6bc54109 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.443508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:26.453317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.454232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.454801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.455286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.456135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.457688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:26.468826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:26.469887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:26.470792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:26.478626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 421/857] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:26.479901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.482119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f4113-e86c-43c1-be2b-0c0242699040 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.530247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.531465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.531577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.531776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.532443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:26.532737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.534224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.549118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 437/858] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:26.550095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.597610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72bd9e-6e4c-4065-8f59-bf728f732ffd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.606935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:26.637546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.638688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.639265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.639710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.640540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:26.641020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.642719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.652829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 422/859] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:26.654103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.657652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4323-aec3-404e-91fd-0e30055f2782 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.693494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.694125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.694398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.694671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.706163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.707583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.716939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:26.725832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:26.726798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:26.727733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:26.727733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:26.727733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:26.732026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 438/860] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:26.733070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:26.768326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41406f2-d0b5-422c-b189-c4aed9e49667 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.815732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.817135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.817736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.818481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.819716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:26.820380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.822881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a054f-9a5c-4bc6-b3f6-bdb37c8bb590 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.838318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 423/861] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:26.840935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a5eb-30d3-40ed-b023-7f1ca3cf4238 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:26.846983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:26.880190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bde6-dcee-4236-9d7e-91aee00f6bed None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.880554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bde6-dcee-4236-9d7e-91aee00f6bed None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.880789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bde6-dcee-4236-9d7e-91aee00f6bed None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.886456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 424/862] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:26.887256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:26.887935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.888187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.888577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.889061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:26.889388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:26.889485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:26.889689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:26.889793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:26.891091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:26.906725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 439/863] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:26.910318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:26.911986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:26.912421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.007162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 425/864] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:27.034594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b82fc-fde2-46b5-8769-ed8638a7b466 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.071730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:27.072437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.072716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.072985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.081033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:27.082567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:27.095908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:27.096965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:27.097925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 440/865] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:27.103122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.159391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:27.212153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ab4c-84e8-4a95-ab86-28a969ce7de0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.247939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:27.248665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.248949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.249227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.249957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:27.250263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:27.251728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:27.276534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 426/866] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:27.281811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.290047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.309583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:27.404315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bc178-1bd2-49dc-b860-eb0c68533a8f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.410511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871ff23-1698-4eac-aeeb-c5eb012d2abd None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.415134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:27.443336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:27.444111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.444518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.444880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.445691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:27.447211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:27.453581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:27.454269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.454584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.454858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.464908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 427/867] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:27 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:27.551586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:27.553865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:27.583418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:27.667711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 441/868] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:27.674047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.792498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96c70-856e-49ad-b2a8-fbdd6763a903 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.832049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:27.833122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.833594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.834061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.846762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:27.848572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-c57ac804-25ad-4660-8da5-80ae723bd16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:27.864344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:27.865311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:27.866226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:27.866226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:27.866226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:27.868362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 428/869] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:27.874529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:27.988876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275bacc-3ac0-42f1-a2fb-0630429efa63 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:27.989747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeb37-bc89-40a9-9d1d-211123adb273 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.990039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeb37-bc89-40a9-9d1d-211123adb273 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.990310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faeb37-bc89-40a9-9d1d-211123adb273 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:27.991971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 429/870] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:27.997140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:27.997569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:27.997918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.014819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.019391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:28.021028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:28.026630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.027285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.027562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.027831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.028501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:28.028850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.030466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.034125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.044599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 442/871] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:28.051224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.100896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 430/872] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:28.173857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b42ec-c3dc-4a7a-b90d-d817e251f427 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.209383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.210011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.210287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.210554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.217948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.219416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:28.230510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:28.231411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:28.232509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 443/873] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:28.239648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.344622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7c0f2-41dd-410e-ba9f-cb8c4ce1d5f6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.380217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.381163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.381635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.382143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.383055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:28.383576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.385299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.397986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 431/874] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:28.404765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.429495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.440150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.462255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.522092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce8820-317a-47ce-a46a-3e8092eaa715 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.540260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389edc32-77d1-414e-9dec-2549d1ad5062 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.558603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.559625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.559990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.560415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.561262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=9aabe3099e2147e29be189622d916500) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:28.561666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.563300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.579050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 444/875] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/role_assignments?user.id=9aabe3099e2147e29be189622d916500&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:28.587877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.588542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.588821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.589094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.589689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.591268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:28.601375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:28.602461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:28.603348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 432/876] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:28.609363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.721446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786b2a1-f23e-4fda-aecb-c29a3f0f264a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.731159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.761342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.762325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.762904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.763547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.764186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:28.764487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.765864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6e699-48f0-42ec-9636-2c10c342a318 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.771733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 445/877] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:28.778454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:28.855944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:28.889841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cab33e-db89-4cb4-a647-a1a879cf151e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:28.891628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd921-31fd-4e85-a872-bed17752c751 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.891912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd921-31fd-4e85-a872-bed17752c751 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.893044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd921-31fd-4e85-a872-bed17752c751 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.894697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 446/878] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:28.900066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10e1a3-75ea-4d46-8779-6a0c69d8bb1a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.900518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10e1a3-75ea-4d46-8779-6a0c69d8bb1a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.900984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10e1a3-75ea-4d46-8779-6a0c69d8bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.920247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8b10e1a3-75ea-4d46-8779-6a0c69d8bb1a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:28.921771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8b10e1a3-75ea-4d46-8779-6a0c69d8bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:28.931579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:28.932711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:28.933001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:28.933281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:28.933923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:28.934227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:28.936730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:28.943520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 433/879] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:29.007213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 447/880] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:28 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:29.267979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:29.381110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c570cb-04f0-4969-9016-a06844c6b734 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:29.417864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:29.418745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:29.419404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:29.419926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:29.420836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:29.422457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:29.425416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3dd95-fae9-42d5-8b29-f5fecbf75ef6 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:29.425711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3dd95-fae9-42d5-8b29-f5fecbf75ef6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:29.425991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3dd95-fae9-42d5-8b29-f5fecbf75ef6 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:29.426985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:29.427985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 448/881] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:29.433903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:29.434211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:29.434552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:29.443522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 434/882] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:29 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:29.455630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:29.457315 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:29.539870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 449/883] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:29.811403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:29.921622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3548-cfa6-46bf-a0ec-7bba871c1cef None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:29.956606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:29.957248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:29.957529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:29.957808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:29.958397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:29.959862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:29.969401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:29.970395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:29.971280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 435/884] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:29.976831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:30.004669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:30.081143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244e85-d1cd-4175-ad99-67e835b4d2ea None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:30.116318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:30.116933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.117210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.117480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.118114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:30.118411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:30.119832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:30.125533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 450/885] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:30.130892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:30.197033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:30.246067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e51e90-beac-4e85-9d69-c281bde1989c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:30.282790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:30.283768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.284246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.284743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.285566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:30.286064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:30.287677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:30.293608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 436/886] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:30.299498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:30.395686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:30.406133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b5976-9b1d-4b91-965e-b266333513c7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:30.444318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:30.445002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.445283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.445553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.446261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] RBAC: Adding query filter params (interface=public, service_id=f561c40cdaf74c08b5a3e8068e6bc829, region_id=RegionOne) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:30.446564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:30.448080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117382a-0994-4d90-8850-e1701886cd11 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:30.454319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 451/887] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:30 2022] GET /identity/v3/endpoints?service_id=f561c40cdaf74c08b5a3e8068e6bc829&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:30.460495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:30.567202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760ea5-64de-4abb-8f4f-1c78a041d7d5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:30.591799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:30.604937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:30.605655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.605989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.606286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.606992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:30.608383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233404a-dd23-4dda-8994-21e2fe77dbaa None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:30.619194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 437/888] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:30 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:43:30.800504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7a94-d24f-4d95-abfc-e17649495f61 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.801156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7a94-d24f-4d95-abfc-e17649495f61 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.801632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7a94-d24f-4d95-abfc-e17649495f61 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.803458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 452/889] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:30.808868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.809163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.809444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.828060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:30.829308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:30.881090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3907-5c61-4d77-be4c-f3d5a3e5ebb2 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.881408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3907-5c61-4d77-be4c-f3d5a3e5ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.881640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3907-5c61-4d77-be4c-f3d5a3e5ebb2 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.883231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 453/890] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:30.888093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:30.888369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:30.888680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:30.900180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 438/891] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:30.908072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:30.909298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:30.979987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 454/892] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:31.173925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.248558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.280890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429e106-aeac-420d-8361-8c60a809e1a0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:31.298385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:31.317518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:31.318160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.318440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.318712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.329550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:31.334401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:31.335555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:31.336444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 439/893] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:31.342987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.352500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:31.376089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dde661-8c0b-48d2-8d71-3185b9228223 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:31.419249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:31.420348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.421099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.421480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.422492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:31.424307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a620f75-3d7d-4f4f-bf67-887d4734001f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:31.457430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 455/894] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:31 2022] POST /identity/v3/services => generated 241 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:31.477124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1868784-c2ff-4f5d-9043-7130f5edd0d7 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.477520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1868784-c2ff-4f5d-9043-7130f5edd0d7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.477726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1868784-c2ff-4f5d-9043-7130f5edd0d7 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.483606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 456/895] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:31.485453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.485742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.486076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.490434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b8a2a-914c-4590-b4de-3239b03b2fb4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:31.506140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:31.507620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:31.547029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:31.547775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.548130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.548521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.549247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:31.549614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:31.551215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:31.562381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 440/896] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:31.567695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.599687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 457/897] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:31.609952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:31.682927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e3e33-0dfd-44ab-80e7-e0870f34c4af None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:31.719405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:31.720047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.720326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.720597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.732752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:31.734297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:31.750942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:31.752036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:31.753218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:31.753218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:31.753218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:31.753218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 441/898] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:31.758806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.838283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:31.864208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71773d19-7324-46a6-bce9-92826b022b0a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:31.898705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:31.899952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:31.900590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:31.900871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:31.901133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:31.901791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:31.902090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:31.903756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:31.916300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 458/899] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:31.921807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.013944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.029084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1ecb-3a76-4e00-a6f4-cdac98c8f8ab None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.036382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155f1dc-db32-4667-858e-168f85b13aae None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.079179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.079820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.080095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.080361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.080964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.082342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.083915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.084951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.085752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.086091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.089469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:32.093170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:32.094333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:32.095326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 442/900] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:32.100973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.101975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.105960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:32.124609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:32.125615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:32.126565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:32.133048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 459/901] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:32.135426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.228351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc10d67-d61a-46bd-be5f-7c628871c23d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.238825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd00a-7a1a-421b-bb5f-5e344f8a445f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.268971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.269934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.270475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.270953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.271963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:32.272519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.273969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.274450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.274763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.275162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.275519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.276342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:32.276763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.278321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.286073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 443/902] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:32.287234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.302447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 460/903] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:32.303556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.309580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.380132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.425587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ad6d-e86f-45d9-8eac-b2a587c61422 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.448892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58605-48b0-42e4-80e2-d1b4393ca632 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.472182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.473222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.473597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.473965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.474689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:32.475119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.477071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.488510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 444/904] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:32.490401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.497653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.498368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.498642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.498910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.568831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.570325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bdb04-7aaa-43a8-b163-606858bdc7d9 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.580311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.605183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c30fa-8dc1-470c-9959-157b79ce6ec4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.627829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.642853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.644289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.644577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.644854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.645579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] RBAC: Adding query filter params (interface=public, service_id=1d66d7d5f896484290909690fb63f102, region_id=RegionOne) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:32.645888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.647392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f534950-7bc9-414d-9365-d38c0902b2ce None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.663767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 445/905] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/endpoints?service_id=1d66d7d5f896484290909690fb63f102&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:32.664878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf34d-1c08-4ec4-afee-c71053e1aa51 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.665174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf34d-1c08-4ec4-afee-c71053e1aa51 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.665449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbf34d-1c08-4ec4-afee-c71053e1aa51 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.671212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 446/906] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:32.671212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf34d-1c08-4ec4-afee-c71053e1aa51 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.687059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 461/907] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:32.687251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.687456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.687749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.706847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:32.708073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:32.780861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 462/908] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:32.788511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:32.791159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbf34d-1c08-4ec4-afee-c71053e1aa51 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.827453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.828406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.828884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.829352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.830179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.831796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.839348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 447/909] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:43:32.894503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:32.914252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262bd8-80a9-4af4-9f45-2e110fc943ce None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:32.954802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:32.955514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:32.955785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:32.956066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:32.969129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:32.970652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:32.989499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:32.991291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:32.992858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:32.992858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:32.992858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:32.992858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 463/910] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:32.997464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.010085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.084432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.116391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4740fa-8a22-425f-ac94-2c815d330da9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.152751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.153873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.154357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.154831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.155832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:33.156330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.157953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.171296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 448/911] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:33.187032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.204345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d6e2-8670-425c-abf1-81495c06943e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.253353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.254194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.254494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.255146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.255914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.257702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.261770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.267042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:33.271809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:33.272994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:33.273847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 464/912] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:33.279023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.310995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e5db6-3b89-4e77-b213-6994ecb7bc61 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.349287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.350116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.350529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.350916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.358646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.360380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:33.373304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:33.374590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:33.375578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 449/913] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:33.383108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.398546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9af83-0d86-44e9-bc11-19e033bdaef5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.437462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.438353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.438630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.438904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.439870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:33.440293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.442067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.449255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 465/914] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:33.454832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.499319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9566d-62b8-445d-8e6b-70eff2fce719 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.529601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.539234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.542708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.543659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.544517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.545388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.546308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:33.546787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.548573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.562851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 450/915] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:33.569145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.586739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fca53c-7a25-4adf-8426-4bf12ed43f00 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.623931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.624457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.624734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.625005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.625643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:33.625949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.627397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc010a-9ce7-4e56-af45-b3f8858e7c94 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.632595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 466/916] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:33.638651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a616f-bf8c-469d-8558-10c71c5ae901 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.638934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a616f-bf8c-469d-8558-10c71c5ae901 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.639234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a616f-bf8c-469d-8558-10c71c5ae901 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.640508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 467/917] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:33.641185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a616f-bf8c-469d-8558-10c71c5ae901 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:33.682429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906d2a1-f829-4edb-a2fc-7af68d95dfa6 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.718878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.719525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.719803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.720800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.720920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] RBAC: Adding query filter params (role.id=5484d57a7112428dbe2a7d219ed1d0ab, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=9aabe3099e2147e29be189622d916500) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:33.721179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.722617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a5367-e7ff-4c12-a972-e7b799cbdecf None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.735440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 451/918] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:33 2022] GET /identity/v3/role_assignments?user.id=9aabe3099e2147e29be189622d916500&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=5484d57a7112428dbe2a7d219ed1d0ab => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:33.737930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.738441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.738921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.761297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a616f-bf8c-469d-8558-10c71c5ae901 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:33.762250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:33.763789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:33.803310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.815779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:33.817299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:33.817861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:33.818340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:33.819533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:33.819899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:33.822678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:33.846507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 452/919] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:33.862797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 468/920] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:34.115845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:34.225056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55555046-baee-44c0-b4d2-b4aa46f03282 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:34.272742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:34.273391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.273647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.273923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.274570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:34.275983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:34.285645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:34.286692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:34.287568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 453/921] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:34.293098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:34.405773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efa99a-d6f3-45d9-a127-d649b43e4d44 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:34.442417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:34.443049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.443329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.443597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.444224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:34.444518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:34.445889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:34.452842 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 469/922] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:34.458399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:34.504652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:34.565597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc4c7-8407-4863-a406-3ec76eb14ef2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:34.601317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:34.601935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.602297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.602574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.603239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:34.603545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:34.604923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:34.610419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 454/923] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:34.616377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:34.666955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:34.726388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deec0b-fa9a-464f-ad86-cf4a94958ec0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:34.761979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:34.762659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.763080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.763450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.764447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] RBAC: Adding query filter params (interface=public, service_id=489924bd048c4f10833c74f673b1791e, region_id=RegionOne) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:34.764880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:34.766894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f87b8e-0a73-4c15-b7b3-0deaa6332385 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:34.773797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 470/924] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:34 2022] GET /identity/v3/endpoints?service_id=489924bd048c4f10833c74f673b1791e&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:34.779638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:34.888081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c314ab-a462-40a0-b4cd-277dadcad605 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:34.890584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:34.926276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:34.926814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.927121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.927395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.928006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:34.929379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:34.931709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4367-594a-4fbd-af9c-bb3c94d06c72 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.932003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4367-594a-4fbd-af9c-bb3c94d06c72 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.932277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff4367-594a-4fbd-af9c-bb3c94d06c72 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.935127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 471/925] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:34.936241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 455/926] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:34 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:43:34.940134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:34.940420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:34.940700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:34.961991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:34.963273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:35.047161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 472/927] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:35.314007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:35.423846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f9bf-a1b5-423f-afe6-6d5fb32ca843 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:35.459224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:35.459875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.460156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.460434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.461023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:35.462467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:35.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:35.473618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:35.474800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 456/928] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 30 00:43:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:35.479579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:35.523888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:35.583782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ee603-5e1d-47eb-824b-5fed451a7886 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:35.623153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:35.623806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.624079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.624344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.624981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:35.625277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:35.626651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eecef-b737-4408-a2af-2218dda4c7aa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:35.632117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 473/929] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:35.637993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:35.695430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:35.744100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aadd06-5c48-425b-b010-97703f172d07 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:35.773371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1def-fe6e-4612-a2f1-31c825f41042 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.773699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1def-fe6e-4612-a2f1-31c825f41042 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.773925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1def-fe6e-4612-a2f1-31c825f41042 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.775568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 474/930] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:35.780787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be739d-295d-49e6-8747-1ac4a5da3823 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.781065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be739d-295d-49e6-8747-1ac4a5da3823 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.781334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be739d-295d-49e6-8747-1ac4a5da3823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.782355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:35.783244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.783762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.784235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.785114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:35.785608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:35.787203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:35.797745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 457/931] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:35.799295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:35.800669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-26be739d-295d-49e6-8747-1ac4a5da3823 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:35.801856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-26be739d-295d-49e6-8747-1ac4a5da3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:35.895168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 475/932] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:35.902265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:35.909926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66604bf4-5f92-48d5-8d1f-8ee95d402524 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:35.945473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:35.946572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.947082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.947577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.948508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:35.950165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:35.956332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaa4a7-998b-4d1b-9a2a-ac3d351ad05b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.956616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaa4a7-998b-4d1b-9a2a-ac3d351ad05b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.956884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aaa4a7-998b-4d1b-9a2a-ac3d351ad05b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.958470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 476/933] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:35.963381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:35.963603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:35.963873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:35.977591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 458/934] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:35.984535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:35.985763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:36.019434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.072009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 477/935] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:36.181121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.291220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29f51-5b47-4bca-b5ee-6a2f921afba8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.327938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.329411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.329940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.330424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.339714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.346526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:36.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:36.354598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:36.355573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 459/936] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:36.361694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.380418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.483693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3cd1e-630f-4d0a-a378-e6b74f4ed881 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.495072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2015fb-50ad-47a8-9799-6fff63c37d9c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.529721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.530975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.531740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.532637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.533792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.535525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:36.537195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.537809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.538143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.538421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.539108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:36.539411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.539873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING keystone.server.flask.common [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:43:36.540766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:36.555439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 460/937] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:36.557067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.564448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.sql.core [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:43:36.564448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:43:36.564448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [parameters: {'id': '092ed446f543465e97b68330433caf4e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:43:36.564448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105787) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 092ed446f543465e97b68330433caf4e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 092ed446f543465e97b68330433caf4e. Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:43:36.582295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application result.read() Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 00:43:36.583271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application yield resource Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise self.value Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application session.commit() Aug 30 00:43:36.584238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:43:36.585118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:43:36.585991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application result.read() Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:43:36.586778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [parameters: {'id': '092ed446f543465e97b68330433caf4e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.589154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:36.590141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 092ed446f543465e97b68330433caf4e. Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.591522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 478/938] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Aug 30 00:43:36 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:36.593655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.621715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.656951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.686084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cce44-536f-4180-bbb9-a1cfa1ad5f53 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.730271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.730910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.731258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.731688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.734250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f2d0-ef63-4081-9ca0-61705536d872 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.749177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.751641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:36.770589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.771265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.771545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.771815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:36.774196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:36.775132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:36.776023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:36.776023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:36.776023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:36.778373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 461/939] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:36.781896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.785407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.787287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:36.804320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:36.805267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:36.806201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:36.810862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 479/940] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:36.811782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.907849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.911535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a742cd6-bf61-408a-9e16-99e8a95d3af2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.938313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c230c2e-0db9-4438-8081-018b1e547e60 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:36.940570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:36.951738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.952478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.952764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.953114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.953670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:36.954035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.955586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ab3db-9ccf-4795-8ca8-b6a973e42d9b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:36.972153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 462/941] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:36.973584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389c77-9bf1-4d31-aa00-de29e8cbd179 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.973941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389c77-9bf1-4d31-aa00-de29e8cbd179 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.974286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389c77-9bf1-4d31-aa00-de29e8cbd179 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.979138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 463/942] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:36.980407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6389c77-9bf1-4d31-aa00-de29e8cbd179 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:36.981940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:36.982715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:36.982995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:36.983308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:36.983987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:36.984344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:36.985729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ce511-0171-49fd-9166-25c5d8cc99ec None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.011042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 480/943] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:37.011042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.011042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.011042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.039037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:37.040718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:37.112037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6389c77-9bf1-4d31-aa00-de29e8cbd179 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.144845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 481/944] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:37.161689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.161689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.161689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.162058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.169787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.171261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:37.182967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:37.183969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:37.185006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 464/945] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:37.192204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.272510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:37.316232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b135b-ae08-469a-8f05-8fca9d196924 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.353694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.354565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.355032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.355463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.356393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:37.356898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.358497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.379821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 482/946] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:37.385678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.421974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.455438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:37.501825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7be79-5366-4ff7-9f77-09ddc0d79c48 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.538031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.538483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b22a-8cee-4a07-ae41-11f082dd877f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.539128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.539128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.539128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.553471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.554942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.566729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:37.570380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:37.571321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:37.572223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:37.572223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:37.572223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:37.572223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 465/947] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:37.578506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.582166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.582820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.583126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.583399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.583996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.585372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:37.595338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:37.597218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:37.598215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 483/948] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:37.605120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.701397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:37.708101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858933-2f2f-4752-a5bb-95d113dfeb1e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.714459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566edf-3322-4f84-90dd-35cea2c0396c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.752329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.752636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.753256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.753496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.753637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.753788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.754178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.754419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:37.754720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.754880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.755668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:37.756099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.756207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.758358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.761925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 484/949] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:37.767377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.773913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 466/950] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:37.782052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.827823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:37.876138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640d71c-af49-4e0a-92ca-e3f0674e8966 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.910217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af2350-995a-4b9b-a276-1e018424c0d8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:37.911230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.911948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.912295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.912688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.913385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:37.913750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.915254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.925344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 485/951] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:37.926305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:37.955865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:37.957043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.957504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.958110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.967512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:37.968972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ffa27f38-33ce-4e27-bb51-15224bf727e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:37.981983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:37.983402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:37.985029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:37.987194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 467/952] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:37.988443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2a9fd-1c84-482e-8d02-5f57522305f9 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:37.988718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2a9fd-1c84-482e-8d02-5f57522305f9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:37.988990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2a9fd-1c84-482e-8d02-5f57522305f9 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:37.990235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 468/953] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:37.990910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2a9fd-1c84-482e-8d02-5f57522305f9 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:38.011311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:38.042761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611114f-f237-465a-bfe3-d20ad5150226 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:38.061456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:38.081178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:38.081792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.082069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.082658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.083029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:38.084415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec69a8-e356-4429-8471-5f644acd5a57 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:38.103776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2a9fd-1c84-482e-8d02-5f57522305f9 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:38.106533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 486/954] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:38.107851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121634-0718-42fd-a0a5-edee88650524 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.108140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121634-0718-42fd-a0a5-edee88650524 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.108470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121634-0718-42fd-a0a5-edee88650524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.140972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-16121634-0718-42fd-a0a5-edee88650524 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:38.142347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-16121634-0718-42fd-a0a5-edee88650524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:38.173355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:38.174049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.174335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.174790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.176731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:38.177229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:38.178796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:38.199189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 469/955] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:38.204661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:38.244128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 487/956] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:38.299465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:38.323871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66ac85-5755-460d-bb7d-60bb09394836 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:38.365054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:38.365720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.365998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.366273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/9aabe3099e2147e29be189622d916500/roles/5484d57a7112428dbe2a7d219ed1d0ab` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.368504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=9aabe3099e2147e29be189622d916500, role_id=5484d57a7112428dbe2a7d219ed1d0ab)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:38.369940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:38.385317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 470/957] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:38 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/9aabe3099e2147e29be189622d916500/roles/5484d57a7112428dbe2a7d219ed1d0ab => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:43:38.527232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121634-0718-42fd-a0a5-edee88650524 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:38.632099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121634-0718-42fd-a0a5-edee88650524 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:38.634527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:38.671674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:38.672680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.673114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.673529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] PATH_INFO: `/v3/services/compute` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.674266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:38.675829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:38.685377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:38.686712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:38.687918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:38.690474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 488/958] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:38.696424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:38.803588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957a079-ff04-44fa-9b6c-eeb74b513f5f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:38.838905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:38.839550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:38.839827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:38.840095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:38.840780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] RBAC: Adding query filter params (name=compute) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:38.841077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:38.842878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d10734-32de-4ab9-9309-ffea2c4a5860 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:38.844043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:38.854967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 471/959] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:38.856479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:38.977939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae8f85-ab1b-4798-9040-23ddf0b88ffe None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:39.012523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:39.013130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:39.013410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:39.013678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:39.014307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:39.014601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:39.016001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c4b2-6099-45d8-9cbc-4071e469a3f2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:39.020893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 489/960] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:39.132877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e1ea-ce8e-42b5-a925-4ad710a1e512 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:39.133213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e1ea-ce8e-42b5-a925-4ad710a1e512 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:39.133431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1e1ea-ce8e-42b5-a925-4ad710a1e512 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:39.134955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 472/961] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:39.140725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:39.141340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:39.141820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:39.162174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:39.163807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:39.192948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:39.239225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 490/962] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:39.520485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e1ea-ce8e-42b5-a925-4ad710a1e512 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:39.651713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1e1ea-ce8e-42b5-a925-4ad710a1e512 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:39.689974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:39.690590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:39.690871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:39.691177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:39.691866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:39.693291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:39.695851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING keystone.server.flask.common [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:43:39.709245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.sql.core [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:43:39.709245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:43:39.709245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [parameters: {'user_id': 'd811b463c0ee4de3bbb427f48213d301', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:43:39.709245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105788) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result.read() Aug 30 00:43:39.716673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:43:39.717868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application yield resource Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise self.value Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application session.commit() Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:43:39.718912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:43:39.719838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:43:39.720746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application result.read() Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:43:39.721616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd811b463c0ee4de3bbb427f48213d301', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:39.722463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 00:43:39.728692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:39.729848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 473/963] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 30 00:43:39 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:39.730344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:39.833659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f400bb4-ade7-4dfd-a879-1e397cfe9ed8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:39.868916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:39.869549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:39.869825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:39.870098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:39.877707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:39.879189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-19d90671-43ff-4951-ac38-e158c1b5e7b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:39.891292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:39.893461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:39.895759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 491/964] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:39.903657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:39.969418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:40.013253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a33e-9f40-4043-88fe-4af9d412b24b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:40.050456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:40.051097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.051381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.051656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.052413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:40.052786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:40.054361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9abe7-a8df-4a30-8a36-888b83466e0f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:40.095751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 474/965] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 30 00:43:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:40.184883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eede36-d54d-442a-9460-83924c410c96 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.185306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eede36-d54d-442a-9460-83924c410c96 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.185598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eede36-d54d-442a-9460-83924c410c96 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.187434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 492/966] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:40.193559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224e0b-46ea-45b8-9c75-673979aa5a28 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.193854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224e0b-46ea-45b8-9c75-673979aa5a28 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.194107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224e0b-46ea-45b8-9c75-673979aa5a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.215972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f5224e0b-46ea-45b8-9c75-673979aa5a28 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:40.217970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f5224e0b-46ea-45b8-9c75-673979aa5a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:40.295126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 475/967] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:40.392613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075aafb-edaa-4ad1-ae3d-a70c2d37e23b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.392967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075aafb-edaa-4ad1-ae3d-a70c2d37e23b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.393138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075aafb-edaa-4ad1-ae3d-a70c2d37e23b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.394664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 493/968] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:40.399874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.400157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.400423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.419655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:40.420860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:40.471480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:40.497798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 476/969] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:40.562375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075aafb-edaa-4ad1-ae3d-a70c2d37e23b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:40.589360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:40.672096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075aafb-edaa-4ad1-ae3d-a70c2d37e23b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:40.706366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:40.707013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.707300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.707568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.708150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:40.709549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:40.718992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:40.719981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:40.720846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:40.723333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 494/970] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:40.726216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:40.762763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:40.830622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c232a95-8280-4005-846f-ea3843b234ea None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:40.856727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:40.870789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:40.872051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.872343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.872619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.873257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:40.873567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dea38-46df-45fa-ab9e-e4699b2c0ee7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:40.873677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:40.874945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:40.885865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 477/971] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:40.886730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:40.911717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:40.912357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:40.912638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:40.912912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:40.924398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:40.929183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:40.930111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:40.930963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:40.935344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 495/972] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:40.936106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:40.937250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.001341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ecb77-2f1d-415f-bf57-257b5efa59d1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.035859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.036459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.036737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.037009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.037631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:41.037933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.039132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77b641-6a30-48fb-8e7a-6fd32a2a86e4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.039330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cd3c5-bd7e-4c0f-943d-c7bf973f2860 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.044912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 478/973] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:41.046081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa656e-ba92-492e-bc1a-ee39bf4d2254 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.046368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa656e-ba92-492e-bc1a-ee39bf4d2254 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.047104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa656e-ba92-492e-bc1a-ee39bf4d2254 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.050432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 479/974] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:41.051300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa656e-ba92-492e-bc1a-ee39bf4d2254 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.076406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.077011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.077283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.077550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.078209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:41.078503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.079907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c3f74-6084-4e6e-9dd4-801ac31feeea None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.088539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 496/975] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:41.089493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.089805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.090061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.107815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:41.108990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:41.110958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.157632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa656e-ba92-492e-bc1a-ee39bf4d2254 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.164696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.178776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 497/976] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:41.179591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.193934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.194928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.195237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.195672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.196368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] RBAC: Adding query filter params (interface=admin, service_id=489924bd048c4f10833c74f673b1791e, region_id=RegionOne) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:41.196674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.198074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.203273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 480/977] 10.222.0.17 () {56 vars in 1249 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/endpoints?service_id=489924bd048c4f10833c74f673b1791e&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:41.208870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.283913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f3723-20f6-454c-8148-425efb0fdac9 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.313274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a679e-494a-4804-9d5f-b5ad8e12131b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.320765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.321403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.321682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.321950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.329000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.333647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.335935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.350452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.351094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.351433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.351712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:41.352146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:41.353051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:41.353920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:41.353920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:41.353920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:41.354100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.354100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.358322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 498/978] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:41.359212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.371064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 481/979] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:43:41.451951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.483376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff545-da89-4f5f-be6a-3eda2d84c3d3 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.490811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.522944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.523620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.523899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.524168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.524809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:41.525104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.526480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.540851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 499/980] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:41.546701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.567922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0806b6-fad4-4e6e-be47-c0b725af2b9a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.600453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.603723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.604538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.604853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.605170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.605875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.607324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.623135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 482/981] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:41 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:43:41.656766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3b1e3-2567-421c-ba31-cacbee7dc5e1 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.704568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.705212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.705612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.705880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.714374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.715994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:41.731581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:41.732664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:41.733609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 500/982] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:41.740400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:41.843581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:41.850233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6959-54da-43b6-9ee5-cbea9ac7e221 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:41.885743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:41.886367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:41.886641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:41.886909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:41.888079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:41.888191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:41.889677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:41.907699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 483/983] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:41.913727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:42.026749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26d85-4dc8-411b-8ec7-dac197fcc41f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:42.034927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:42.066975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:42.067671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:42.068014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:42.068456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:42.145171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:42.146732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:42.262446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:42.263638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 501/984] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:41 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:42.381591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64981b-b017-4151-98fe-d36da8701636 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:42.429903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:42.430559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:42.430828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:42.431151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:42.443175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:42.444661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:42.465286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:42.466745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:42.468164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:42.468164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:42.468164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:42.468164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 484/985] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:42.474385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:42.482801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:42.578485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa19f1d-f113-4723-bdd8-e3ea8ecaf185 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:42.613200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:42.613811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:42.614066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:42.614344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:42.614983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:42.615316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:42.616719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:42.628764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 502/986] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:42.634399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:42.638546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:42.740110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42c194-e04f-49a1-94a1-9704fe19b732 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:42.778659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:42.779368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:42.779607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:42.779883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:42.787508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:42.788970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:42.802012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:42.803098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:42.804033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:42.807210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 485/987] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:42.810334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:42.885463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:42.926890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b8155-bd63-437c-b24a-5d273aa8df76 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:42.962372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:42.963173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:42.963551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:42.963922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:42.964720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:42.965144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:42.967595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef6f1-c4a0-4d49-88ab-be06ef7262cd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:42.985566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 503/988] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:42.986953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:43.053783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:43.097123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cd89c-50db-40ec-8c68-29a8b1652b2e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:43.132005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:43.132612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.132886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.133156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.133868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] RBAC: Adding query filter params (role.id=5484d57a7112428dbe2a7d219ed1d0ab, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=9aabe3099e2147e29be189622d916500) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:43.134180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:43.136069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7880a-3d7a-4425-8a42-3154d645f109 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:43.142555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 486/989] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:42 2022] GET /identity/v3/role_assignments?user.id=9aabe3099e2147e29be189622d916500&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=5484d57a7112428dbe2a7d219ed1d0ab => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:43.306757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a07-ca2c-4733-8fe1-eec6c8dae76b None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.307210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a07-ca2c-4733-8fe1-eec6c8dae76b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.307576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1269a07-ca2c-4733-8fe1-eec6c8dae76b None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.308955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 504/990] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:43.315344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adcfb0-99bb-4998-ac3d-acd418b44310 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.315650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adcfb0-99bb-4998-ac3d-acd418b44310 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.315909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adcfb0-99bb-4998-ac3d-acd418b44310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.336096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c0adcfb0-99bb-4998-ac3d-acd418b44310 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:43.337430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c0adcfb0-99bb-4998-ac3d-acd418b44310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:43.423042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 487/991] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:43.593947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede86215-f904-40ec-b523-b20d262a9328 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.594465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede86215-f904-40ec-b523-b20d262a9328 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.594848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede86215-f904-40ec-b523-b20d262a9328 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.596402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 505/992] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:43.601794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.602115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.602340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.621136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:43.622469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:43.679591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:43.681291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede86215-f904-40ec-b523-b20d262a9328 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:43.703118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 488/993] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:43.787772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede86215-f904-40ec-b523-b20d262a9328 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:43.814237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:43.824046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:43.824700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.825034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.825295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.825872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:43.827380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:43.836838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:43.837811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:43:43.838661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:43.840156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 506/994] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 30 00:43:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:43.845502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:43.956098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf460e4-0add-41fb-8d0f-be4956484df5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:43.967525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:43.992176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:43.992818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:43.993094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:43.993349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:43.993974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:43.994269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:43.995663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.005537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 489/995] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:44.006779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.069788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:44.077311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e23c1-eadc-49a7-9d80-d3955ee11011 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.112815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.113550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.113937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.114212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.114788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.116240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.122890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06d7f4-fe76-414f-91a6-cea93d31f5f6 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:44.125738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:44.126711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:44.127959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 507/996] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:44.132540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.165455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.166339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.166866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.167174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.168110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:44.168646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.171407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.182596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 490/997] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 30 00:43:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:44.184504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.194568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:44.247635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2e1b3-072a-4c3b-832b-f171b81895b4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.282621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.283266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.283548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.283820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.284447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] RBAC: Adding query filter params (name=compute) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:44.284753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.286128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.290805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 508/998] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:44.296758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.303289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241a94-5135-40a5-ae44-8afaf9cca2d6 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.321277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:44.342135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.342750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.343052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.343328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.343990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.345355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.364092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 491/999] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:44.415326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22c967-90cc-4bb2-a512-afe02d693127 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.436048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:44.463146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.463811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.464088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.464353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.464979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:44.465274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.466762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.473216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 509/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:44.479154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.588631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08156018-b5b4-4a95-8d5e-6ac81a0b6a1a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.623492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.624110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.624395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.624664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.625367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] RBAC: Adding query filter params (interface=public, service_id=c745e842acfd496cbaa812b9b45495ef, region_id=RegionOne) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:44.625671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.627112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246812f-0d86-47b7-a1a2-691cca997a43 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.632760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 492/1001] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:44 2022] GET /identity/v3/endpoints?service_id=c745e842acfd496cbaa812b9b45495ef&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:44.638364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:44.742094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ddae7-628a-4c5e-8acb-3504f30871d6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:44.776512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:44.777118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:44.777390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:44.777654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:44.778304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:44.779712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea8971-9747-4c3e-981d-4b88305af596 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:44.786227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 510/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:44 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:43:45.071637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300f9a-251f-427a-8caa-7dac72c4893c None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:45.072211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300f9a-251f-427a-8caa-7dac72c4893c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:45.072211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300f9a-251f-427a-8caa-7dac72c4893c None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:45.073700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 493/1003] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:45.078893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:45.079458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:45.079458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:45.097972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:45.099177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:45.133528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:45.170026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 511/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:45.431804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2300f9a-251f-427a-8caa-7dac72c4893c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:45.481904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:45.544895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2300f9a-251f-427a-8caa-7dac72c4893c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:45.580033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:45.580727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:45.581006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:45.581279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] PATH_INFO: `/v3/services/image` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:45.581855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:45.583261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:45.592407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:45.593416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:45.596106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 494/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 30 00:43:45 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:45.599538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:45.699874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc301a-2ba9-406a-91a0-ff41c2d262ae None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:45.734568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:45.735601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:45.736086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:45.736582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:45.737461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] RBAC: Adding query filter params (name=image) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:45.737961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:45.739623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3fbf0-6ff1-4245-a4fe-526b2d40694f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:45.745105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 512/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:45.751572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:45.859442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7b8e-95cc-4968-b0f8-acb59a60277e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:45.861949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:45.896510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:45.897123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:45.897406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:45.897677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:45.898297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] RBAC: Adding query filter params (type=image) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:45.898593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:45.899993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe837d-b7c3-4553-9f88-31d0c2ec5571 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:45.904759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 495/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:46.307052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d90e7-7092-4a92-b443-d578646d8dbf None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.307637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d90e7-7092-4a92-b443-d578646d8dbf None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.307637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d90e7-7092-4a92-b443-d578646d8dbf None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.309470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 513/1008] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:46.314621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bf216-27ea-4ac6-82b8-cce9661d469d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.315101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bf216-27ea-4ac6-82b8-cce9661d469d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.315205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bf216-27ea-4ac6-82b8-cce9661d469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.334179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b04bf216-27ea-4ac6-82b8-cce9661d469d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:46.335447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b04bf216-27ea-4ac6-82b8-cce9661d469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:46.411952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 496/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:46.667345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8da6-c785-4cb1-ae6c-eb87a1d18149 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.667598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8da6-c785-4cb1-ae6c-eb87a1d18149 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.667856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad8da6-c785-4cb1-ae6c-eb87a1d18149 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.669454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 514/1010] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:46.673952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad8da6-c785-4cb1-ae6c-eb87a1d18149 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:46.674759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.675099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.675321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.695075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:46.696271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:46.710163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:46.775383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 497/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:46.780934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad8da6-c785-4cb1-ae6c-eb87a1d18149 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:46.818007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:46.818640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.818915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.819215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.820330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:46.821765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:46.827075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 515/1012] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:46.829594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:46.859174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:46.933291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6c645-eb00-46fb-8445-2a7ef8e8a79d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:46.968779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:46.969518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:46.969696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:46.969956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:46.970644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:46.972154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:47.005417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 498/1013] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:46 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:47.034319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:47.145328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1284e-a8bb-4a26-958d-35fd96597291 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:47.147629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:47.183214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:47.184475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.184764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.185037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] PATH_INFO: `/v3/services/compute` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.185626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:47.187023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:47.196095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:47.197087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:47.197975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 516/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:47.203366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:47.289627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:47.311133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065425a0-ac84-45c1-b83a-c9b8e6be8b00 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:47.345283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:47.345966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.346197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.346484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.347145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] RBAC: Adding query filter params (name=compute) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:47.347452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:47.348840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:47.354131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 499/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:47.360022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:47.473188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58810263-7c58-4a16-a39e-ba0afb824811 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:47.510224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:47.510829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.511201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.511474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.512373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] RBAC: Adding query filter params (type=compute) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:47.512751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:47.514194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0b56d-5587-4d2d-8d2a-a1ca0e3ad814 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:47.519475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 517/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 30 00:43:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:47.525010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:47.629043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b53f3-e570-43cf-bd97-fd8265aea738 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:47.663331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:47.663949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.664222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.664488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.665160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:47.666507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fb9c5-4270-48d9-a76e-afa6adf856d8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:47.683160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:47.699464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 500/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:47.802014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e834-c441-409f-b93b-642418daf564 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.802214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e834-c441-409f-b93b-642418daf564 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.802452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e834-c441-409f-b93b-642418daf564 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.804043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 518/1018] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:47.809735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86366f1-9009-4926-b119-82316d6ec539 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:47.810029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86366f1-9009-4926-b119-82316d6ec539 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:47.810632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86366f1-9009-4926-b119-82316d6ec539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:47.831359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a86366f1-9009-4926-b119-82316d6ec539 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:47.832583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a86366f1-9009-4926-b119-82316d6ec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:47.910824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 501/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:47 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:48.179068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5e834-c441-409f-b93b-642418daf564 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:48.213531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:48.288396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5e834-c441-409f-b93b-642418daf564 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:48.322378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:48.322987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:48.323285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:48.323551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:48.324196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:48.325753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bd79d-8a27-4890-8d93-2f6b4a163db5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:48.343121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 519/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:43:48 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:48.954168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d427a7-581c-479e-ad00-951f7b594621 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:48.954729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d427a7-581c-479e-ad00-951f7b594621 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:48.954729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d427a7-581c-479e-ad00-951f7b594621 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:48.956280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 502/1021] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:48.961348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:48.961765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:48.961904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:48.982157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:48.983486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:49.059997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 520/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:49.348516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d427a7-581c-479e-ad00-951f7b594621 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:49.469651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d427a7-581c-479e-ad00-951f7b594621 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:49.496183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:49.506510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:49.507176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:49.507458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:49.507790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:49.508897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:49.510342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9557d45-0022-4645-b78b-3b33049f08f4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:49.512309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 503/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:49.517952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:49.536682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:49.622058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd71c1-dac9-42e3-b236-16eeaf37fe90 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:49.657191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:49.657828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:49.658112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:49.658379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:49.659060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:49.660500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8f57-63e1-4508-b3ba-663c38298bd3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:49.685570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 521/1024] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:43:49 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:49.737197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895481c4-8b7f-4299-89e7-3887acd579a1 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:49.737637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895481c4-8b7f-4299-89e7-3887acd579a1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:49.737743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895481c4-8b7f-4299-89e7-3887acd579a1 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:49.743115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 504/1025] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:49.745556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581a12b-45c1-4f7e-9a6e-27f480541870 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:49.745852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581a12b-45c1-4f7e-9a6e-27f480541870 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:49.746188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581a12b-45c1-4f7e-9a6e-27f480541870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:49.767229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c581a12b-45c1-4f7e-9a6e-27f480541870 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:49.768563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c581a12b-45c1-4f7e-9a6e-27f480541870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:49.845276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 522/1026] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:50.111403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895481c4-8b7f-4299-89e7-3887acd579a1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.226253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895481c4-8b7f-4299-89e7-3887acd579a1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.274289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:50.274937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.275262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.275531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.286182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373171-c591-4332-a411-2f4e65e6d962 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.286182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373171-c591-4332-a411-2f4e65e6d962 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.286182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373171-c591-4332-a411-2f4e65e6d962 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.286182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 523/1027] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:50.295522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.295522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.295522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.305235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:50.306725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:50.310503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:50.311678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:50.312511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 505/1028] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:50.318736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.330686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:50.331933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:50.384200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:50.411048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 524/1029] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:50.444332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc58-d9bb-43ea-bf7c-e9ec5d16beda None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.482255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:50.483319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.483821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.484296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.485166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:50.485666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:50.487328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:50.499655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 506/1030] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:50.505518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.613612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb220d-f869-4b15-a020-67a1e43101bc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.650867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:50.651875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.652355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.653043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.655483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:50.666694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:50.668513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:50.674031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:50.685971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:50.687201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:50.688176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:50.688176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:50.688176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:50.688176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 525/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:50.693529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.790666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020cd69-64eb-49fe-8f0c-16fb450591dc None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.801605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3d7ae-567a-4434-b075-f68382570dbe None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.807685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:50.828543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:50.829181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.829453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.829722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] PATH_INFO: `/v3/services/image` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.830302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:50.831690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:50.839761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:50.840427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:50.840700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:50.840895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:50.842148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:50.843903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:50.844650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:50.844650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:50.844650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:50.844850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 507/1032] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:50.844918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:50.849223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.860660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 526/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:50.866485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:50.915231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:50.956092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa7f4-4e98-403c-a293-57bc008d9689 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:50.969537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f80e6-d1e2-4667-800f-de2757bc6ef7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.000427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.001509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.002026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.002534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.003563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] RBAC: Adding query filter params (name=image) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:51.004101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.004375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.004999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.005268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.005533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.006569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.012916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.013412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 508/1034] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:51.014297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.018494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:51.026188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:51.027194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:51.028192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 527/1035] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:51.034713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.063816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.124490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27d265-6ea3-494e-af49-2c7e94353a37 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.141569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3793ade-4861-4d35-8346-b4de9811f9fa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.149347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.162435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.163181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.163531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.163869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.164557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] RBAC: Adding query filter params (type=image) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:51.165061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.166357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.171370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 509/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:51.176973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.180246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.180925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.181200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.181489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.182224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:51.182525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.184119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.202669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 528/1037] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:51.208376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.286054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64e3e-fdfb-44c8-9654-37fd86779803 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.295554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.318006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805cd1c-d64c-4c1b-81f1-75ca96aee9fe None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.322674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.323336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.323612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.323877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.324635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] RBAC: Adding query filter params (interface=public, service_id=f18807ef4fec4130aeb7d19a153f26c2, region_id=RegionOne) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:51.324961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.326336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.332011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 510/1038] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/endpoints?service_id=f18807ef4fec4130aeb7d19a153f26c2&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:51.337343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.354435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.355054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.355331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.355650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.387550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.428742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.430126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72173f9c-9c06-4857-a22d-f4bd9a7c4a5f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.451929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94c09-09bc-4e0d-941d-c7292323c490 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.488834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.489447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.489723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.489994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.490591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.491997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.499230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 511/1039] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:43:51.547570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 529/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:51.553687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.594165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.660902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f68092-8359-480d-b0f0-9f3491bd9922 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.661176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f68092-8359-480d-b0f0-9f3491bd9922 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.661446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f68092-8359-480d-b0f0-9f3491bd9922 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.663200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 530/1041] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:51.668149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.668428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.668691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.670058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1951063-f9d6-4030-a53d-93c9cfb3477a None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.691307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:51.692404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:51.705532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.706178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.706456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.706727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.717946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.719419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:51.736387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:51.737480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:51.738413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:51.738413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:51.738413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:51.739286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 512/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:51.743549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:51.762012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.772803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 531/1043] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:51.835555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:51.850096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99122-b8ef-4673-b556-ffd479ccdfa2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:51.885664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:51.886262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:51.886504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:51.886815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:51.887523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:51.887812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:51.889174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:51.901576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 513/1044] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:51.907539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.016763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016295c3-f55c-46e4-819f-943d67f77d26 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.038732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.054461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.055169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.055447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.055713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.062544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.064035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:52.075083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:52.075995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:52.076886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 532/1045] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:52.082145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.136776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:52.153394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efa80-ea34-4489-93be-4c24d3c7c58c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.185561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4154-3e8e-4c77-9fbc-6b83e2d49fe2 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.189828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.190461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.190738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.191024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.201211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:52.205960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:52.206892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:52.208337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 514/1046] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:52.212973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.222707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.223429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.223997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.223997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.224700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:52.225000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.226092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:52.226435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.239452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 533/1047] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:52.245721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.322550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76602f6-8b30-472c-b58d-7dbe0e75deab None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.356870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936991-9c5c-43ad-be2e-73d5a9e90352 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.359879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.360119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.360372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.360642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.361298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:52.361594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.362954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.372131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 515/1048] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:52.373994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:52.377884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.396983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.397590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.397860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.398129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.398849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] RBAC: Adding query filter params (role.id=5484d57a7112428dbe2a7d219ed1d0ab, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:52.399183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.400622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.407089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 534/1049] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=5484d57a7112428dbe2a7d219ed1d0ab => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:52.466025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:52.502067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43821bb1-1e31-4bb0-9065-89821d2aa264 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.541113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.541761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.542041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.542316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] PATH_INFO: `/v3/users/15bc18c73ec642fba3e78c85d0e318a0` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.558046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] RBAC: Authorizing `identity:get_user(user_id=15bc18c73ec642fba3e78c85d0e318a0)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.559726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.561664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 516/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/users/15bc18c73ec642fba3e78c85d0e318a0 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:52.567403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:52.674939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443827-bc81-4721-baf3-abc55ecffb6f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:52.710690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:52.711327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:52.711612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:52.711874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:52.768828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:52.798471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:52.800632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:52.909890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 535/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:52.915601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:53.025188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923be81-f92a-44fd-9d8a-1e85bfb52c30 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:53.057827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:53.068206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:53.068858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:53.069136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:53.069420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:53.070576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:53.072046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f4bd9-586d-4986-a860-2f93d5fa4913 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:53.073852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 517/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:43:52 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:53.079982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:53.165563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:53.191232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ce31c-e5de-4823-9a59-efdb4b6a6ed8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:53.230613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:53.231422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:53.231837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:53.232164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:53.232904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=6a792c10e11846a0b16dd90126661829, user.id=15bc18c73ec642fba3e78c85d0e318a0) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:53.233253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:53.234719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797f5bb-cd30-4366-a62f-6ea7df93a35f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:53.247475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 536/1053] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:53 2022] GET /identity/v3/role_assignments?user.id=15bc18c73ec642fba3e78c85d0e318a0&scope.project.id=6a792c10e11846a0b16dd90126661829&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:53.451533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97999-f268-4797-9ef6-2cf910e0ec03 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:53.452090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97999-f268-4797-9ef6-2cf910e0ec03 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:53.452090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97999-f268-4797-9ef6-2cf910e0ec03 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:53.453463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 518/1054] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:53.459120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:53.459588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:53.459666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:53.480208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:53.481642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:53.565035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 537/1055] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:53.831648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97999-f268-4797-9ef6-2cf910e0ec03 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:53.942946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97999-f268-4797-9ef6-2cf910e0ec03 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:53.979755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:53.980379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:53.980703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:53.980991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] PATH_INFO: `/v3/services/image` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:53.981579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:53.983019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:53.992402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:53.993414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:43:53.994265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:53.998944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 519/1056] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 30 00:43:53 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:54.000277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:54.079600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:54.119828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1ecb6-2b11-4415-b6c9-6ac5d2d89187 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:54.157851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:54.158462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.158792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.159338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.159974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] RBAC: Adding query filter params (name=image) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:54.160271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:54.161669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:54.167044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 538/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:54.172740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:54.237709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:54.282833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8cc33-f9ce-49c4-a6d5-e4d438b6bc0d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:54.317400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:54.318021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.318298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.318570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.319230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] RBAC: Adding query filter params (type=image) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:54.319536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:54.320906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ef580-a7b3-462e-b3f9-3e04f4e6dc20 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:54.326504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 520/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:54.332376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:54.396865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197317-fdc6-4823-86f1-2cb7f66a350d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.397042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197317-fdc6-4823-86f1-2cb7f66a350d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.397312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197317-fdc6-4823-86f1-2cb7f66a350d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.398867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 521/1059] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:54.405015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.405308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.405583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.425344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:54.426583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:54.440459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba261b-5d16-44d1-b2d2-966fb5703ad4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:54.477182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:54.477896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.478059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.478326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.479022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:54.480407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:54.485806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:54.501907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 522/1060] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:54.503582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 539/1061] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:43:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:43:54.768130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:54.840163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:54.906005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079affbc-9a95-4f87-8c84-1f1ad46be8bf None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:54.950449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:54.951143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:54.951450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:54.951727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:54.962515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:54.973361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:54.974364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:54.975274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:54.975934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 523/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:54.982478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.095281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a47d0-d522-40e9-8632-857476797848 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.131238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.132282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.132777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.133254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.134175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:55.134682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.136345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152a161-b31e-4fc5-b4e6-4a056ddbe47d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.150568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 540/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:55.152651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.169033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7ef38-14a5-4556-9892-ec45dd30c5c1 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.169369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7ef38-14a5-4556-9892-ec45dd30c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.169596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7ef38-14a5-4556-9892-ec45dd30c5c1 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.170965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 541/1064] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:43:55.176081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.176369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.176638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.195913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:55.197102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:55.279055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:55.299766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb6841-6898-4862-ad19-07991f85c270 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.306503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 542/1065] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:55.315560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:55.342885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.343548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.343828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.344106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.354725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.356168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:55.370991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:55.372082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:55.373045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:55.373045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:55.373045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:55.373738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 524/1066] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:55.378067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.485781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092d8a7-95a2-4575-80d4-3f6528f8857b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.522100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.522722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.522984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.523290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.523935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:55.524232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.525604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.540101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 543/1067] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:55.546262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.579419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.628016 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:55.663392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43b388-67f2-409a-8b9e-8306265386c9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.687521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8e6ce-7eb0-4688-a42e-878eab49fa26 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.703323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.703960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.704239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.704511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] PATH_INFO: `/v3/users/nova` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.712158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.713591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.724257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.724893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.725170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:43:55.725356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:55.726264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:43:55.727310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:55.727865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.727936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 525/1068] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:55.732507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.741040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:55.745936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:55.746909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:55.747810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 544/1069] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:55.753747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.761436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:55.841169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3652e-679c-4be2-8401-12a5753036af None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.871799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7c54f-0f1c-496b-a056-4ecb3db25e3a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:55.875077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:55.879640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.880191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.880464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.880732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.881447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:55.881743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.883178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.900759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 526/1070] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:55.906172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.911205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:55.911863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:55.912215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:55.912491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:55.913192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:55.913505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:55.915112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:55.924090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 545/1071] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:55.929544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:55.992358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.014337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060a41-47c9-4af9-b9b1-f6101e6424f7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.040135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf309-ee79-4d7f-90b5-9f4c9fd27818 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.049292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.049927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.050202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.050471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.060882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.062330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.076506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.077144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.077422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.077692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] PATH_INFO: `/v3/users/15bc18c73ec642fba3e78c85d0e318a0` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:56.077987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:56.078891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:56.079853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:56.079853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:56.079853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:56.080780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] RBAC: Authorizing `identity:get_user(user_id=15bc18c73ec642fba3e78c85d0e318a0)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.082214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.088597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 546/1073] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/users/15bc18c73ec642fba3e78c85d0e318a0 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:56.088725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 527/1073] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:56.088836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.089373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.124360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.198944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c8d19-c18c-472c-b95f-18376cb0e562 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.199188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a065080-f910-40b9-819a-b87ef7f01643 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.229273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.236154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.236896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.237240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.237569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.238756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.240314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.242031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 528/1074] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:43:56 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:56.242311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.243156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.243629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.244079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.244898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:56.245376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.246968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.248104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.259801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 547/1075] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:56.265724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.360032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cee61a-41e9-4550-84fe-2eb4c6a458da None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.368420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.376601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b8686-a1c6-45eb-a605-eb1253e3a756 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.395861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.396798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.397217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.397628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829/users/15bc18c73ec642fba3e78c85d0e318a0/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.399706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a792c10e11846a0b16dd90126661829, user_id=15bc18c73ec642fba3e78c85d0e318a0, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.401135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.418897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 529/1076] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:56 2022] PUT /identity/v3/projects/6a792c10e11846a0b16dd90126661829/users/15bc18c73ec642fba3e78c85d0e318a0/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:43:56.429670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.430341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.430691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.430974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.439213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.440717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-4ecc0ec4-a1e3-4c7b-bc9e-1cf735cbe9e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:43:56.455702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:56.457417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:56.458405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 548/1077] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:56.462721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.520746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682d7dc-2ebe-4898-b040-ec6ecafc3016 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.521418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682d7dc-2ebe-4898-b040-ec6ecafc3016 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.521610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682d7dc-2ebe-4898-b040-ec6ecafc3016 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.523818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 549/1078] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:56.529946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.530284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.530904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.534073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.551674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:56.552927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:56.591259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7ce6f-e6e6-47e4-86c2-f556aef9b827 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.630269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.630890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.631208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.631480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.632165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:56.632474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.633916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.641622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 550/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:56.648563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 530/1080] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:56.654913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:56.742930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:56.760811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fcdaf-2a4d-453a-b96f-e530d94ba367 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:56.797540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:56.798192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:56.798466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:56.798710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/5484d57a7112428dbe2a7d219ed1d0ab` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:56.800890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=28f8f65dc678403081189e31ef5f8a10, role_id=5484d57a7112428dbe2a7d219ed1d0ab)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:56.802348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:56.819205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 551/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:43:56 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/28f8f65dc678403081189e31ef5f8a10/roles/5484d57a7112428dbe2a7d219ed1d0ab => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:43:56.909279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:57.017949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba6a32-feff-40e9-a1ad-c99998e6f535 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:57.038034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:57.052905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:57.053528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:57.053807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:57.054080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:57.124510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:57.125974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:57.225760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 531/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:43:57.231663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:57.337876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c90e-ee86-4e92-a7b8-d2caf59d3456 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:57.373290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:57.373924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:57.374217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:57.374488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:57.385121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:57.386559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:57.389397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-dc5aa46a-3711-492e-9d92-d638f45e44a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:57.406054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:57.407629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:57.408834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:57.408834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:57.408834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:57.408834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 552/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:57.415285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:57.438296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:57.523559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a024-6896-4c5b-8f8d-cf97cc5ff6d4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:57.558652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:57.559299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:57.559581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:57.559854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:57.560490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:57.560792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:57.562156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07973d6-3640-4dfe-800d-91f2970e0820 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:57.574371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 532/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:58.360422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177bf54-af07-4874-aba6-b0ef4cf95cf4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:58.361015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177bf54-af07-4874-aba6-b0ef4cf95cf4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:58.361015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177bf54-af07-4874-aba6-b0ef4cf95cf4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:58.362457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 553/1085] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:58.368756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45bd62-170e-4b45-ba34-a200216e6270 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:58.369168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45bd62-170e-4b45-ba34-a200216e6270 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:58.369581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45bd62-170e-4b45-ba34-a200216e6270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:58.394381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8b45bd62-170e-4b45-ba34-a200216e6270 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:58.396227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8b45bd62-170e-4b45-ba34-a200216e6270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:58.471680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 533/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:43:58.741113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177bf54-af07-4874-aba6-b0ef4cf95cf4 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:58.805205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106076-013c-489f-8ca6-7baedb4a1cad None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:58.805606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106076-013c-489f-8ca6-7baedb4a1cad None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:58.805913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106076-013c-489f-8ca6-7baedb4a1cad None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:58.807328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 534/1087] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:43:58.812381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:58.812657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:58.812928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:58.831296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:43:58.832483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:43:58.835255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:58.849622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177bf54-af07-4874-aba6-b0ef4cf95cf4 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:58.889022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:58.889677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:58.890002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:58.890299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:58.901103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:58.903437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 535/1088] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:58.906791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:58.907938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:43:58.911858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 554/1089] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:43:58.913983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.014446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3bb62-7269-4e09-b5d6-ca9404fad8a0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.049415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.050012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.050283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.050546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.051228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:59.051523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.052873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.061280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 536/1090] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:59.066997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.124434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.169137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601688a-5ddd-4af9-b3b8-63ef8921dc39 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.172621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.175595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.205395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.206031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.206306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.206578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] PATH_INFO: `/v3/users/15bc18c73ec642fba3e78c85d0e318a0` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.207716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] RBAC: Authorizing `identity:get_user(user_id=15bc18c73ec642fba3e78c85d0e318a0)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.209161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.210769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 555/1091] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/users/15bc18c73ec642fba3e78c85d0e318a0 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:59.217227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.283911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55704-5506-44e7-8ff5-9b1537959ebd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.317761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7352-a78e-4f00-b5b9-397892c1866a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.318126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.318755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.320511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.320511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.329591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:43:59.334686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:59.335702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:59.336767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 537/1092] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:59.342146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.344986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.358253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.358901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.359209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.359478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.406962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.431644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.433156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.446761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f09883-97ea-44a1-9002-08fd95287ab2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.481775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.482480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.482761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.483543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.484201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:59.484501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.485852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.493827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 538/1093] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:59.499764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.536618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 556/1094] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:43:59.542784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.567593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.598792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e150588-8b27-4045-95b3-e8640fb70c58 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.623406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.635405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.636046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.636322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.636654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.645058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce6a13-03a0-45ad-838f-2660a1d997a8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.647574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.649156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:43:59.663871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:43:59.665014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:43:59.665973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:43:59.665973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:43:59.665973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:43:59.665973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 539/1095] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:43:59.672101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.688284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.688921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.689196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.689467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.690597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.692078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.693722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 557/1096] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:59.700532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.781423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3d938-20a8-4ce6-98bd-3e0e6a34a072 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.800144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.809940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8739c-e0ec-4945-b116-344ddf9c4a30 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.817443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.818186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.818463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.818788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.819460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:59.819797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.821162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.833616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 540/1097] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:43:59.839182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:43:59.844609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.845229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.845612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.845771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:43:59.846545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=6a792c10e11846a0b16dd90126661829, user.id=15bc18c73ec642fba3e78c85d0e318a0) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:43:59.846882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:43:59.848752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:43:59.854692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 558/1098] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/role_assignments?user.id=15bc18c73ec642fba3e78c85d0e318a0&scope.project.id=6a792c10e11846a0b16dd90126661829&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:43:59.876534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:43:59.952678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf3133-1e04-4e67-91e6-c361a11d30f9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:43:59.991427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:43:59.992070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:43:59.992346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:43:59.992621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:00.001271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:00.002797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:44:00.017324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:00.018461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:00.019446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 541/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 30 00:43:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:00.024963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:00.142949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef78f2-4d0d-426e-8f3f-3971dfee1a7e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:00.179518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:00.180133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:00.180409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:00.180683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:00.181418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:00.181727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:00.183212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:00.201996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 559/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 30 00:44:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:00.207842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:00.309955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:00.316773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314a968-0209-42c7-b571-98d6fe90d254 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:00.352587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:00.353214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:00.353491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:00.353746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:00.439300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:00.440908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:00.534001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:00.541343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 542/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:00.547346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:00.564055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:00.650856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a823-4f11-4f26-8a1e-7231f6796fd3 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:00.686364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:00.686909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:00.687187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:00.687481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:00.697929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:00.699485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:44:00.714990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:00.716436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:44:00.717394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:44:00.717394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:44:00.717394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:44:00.717394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 560/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:00.722476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:00.822750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc4cb-0c41-4dd7-a99d-4646321a931b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:00.858931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:00.859569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:00.859845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:00.860115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:00.860759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:00.861062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:00.862411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:00.873756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 543/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:44:00.879674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:00.949518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:00.989055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce1b4d-ac68-4c3e-8327-e6affa252b3b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:01.026573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:01.027462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:01.027732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:01.028001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:01.034881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:01.036360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:44:01.048106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:01.049217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:44:01.050137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 561/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:01.055554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:01.113983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:01.158562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861eef9-793e-4f1e-9247-da072da88ddf None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:01.193908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:01.194520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:01.194803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:01.195108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:01.195799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:01.196102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:01.197526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0552-fcf4-492e-b324-a13a6801c67b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:01.210740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 544/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:01.217405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:01.322483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136323-04b4-41ba-afa6-aab4d4ed27ef None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:01.350770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:01.359938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:01.360543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:01.360816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:01.361079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:01.361799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] RBAC: Adding query filter params (role.id=5484d57a7112428dbe2a7d219ed1d0ab, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=28f8f65dc678403081189e31ef5f8a10) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:01.362095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:01.363565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50654b8e-2dc9-4347-a818-4d1550d5cd84 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:01.369629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 562/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:01 2022] GET /identity/v3/role_assignments?user.id=28f8f65dc678403081189e31ef5f8a10&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=5484d57a7112428dbe2a7d219ed1d0ab => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:01.781741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526f2dd-bf08-4418-966f-6d8ca987bf75 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:01.782310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526f2dd-bf08-4418-966f-6d8ca987bf75 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:01.782310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526f2dd-bf08-4418-966f-6d8ca987bf75 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:01.783803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 545/1107] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:01.789109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:01.789578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:01.789741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:01.809839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:01.811064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:01.881812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 563/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:02.158832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526f2dd-bf08-4418-966f-6d8ca987bf75 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:02.265550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526f2dd-bf08-4418-966f-6d8ca987bf75 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:02.282601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:02.301544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:02.302188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:02.302481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:02.302751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:02.303998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:02.305543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358de30-a78a-46fd-9357-d373d412c800 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:02.307149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 546/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:44:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:02.313301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:02.428145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531b718-fae9-47f9-9fc2-297dc789d9e2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:02.448471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:02.470125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:02.470775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:02.471144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:02.471326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:02.472023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:02.474109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ec8ea-4ae5-42ec-937c-8c3a1863283f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:02.499920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 564/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:44:02 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:04.432264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495cfff-a1e6-4963-af96-a241ec72238d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:04.432980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495cfff-a1e6-4963-af96-a241ec72238d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:04.433093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495cfff-a1e6-4963-af96-a241ec72238d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:04.435622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 547/1111] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:04.443457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:04.443920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:04.444009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:04.464443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:04.465659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:04.534512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 565/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:04.818949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495cfff-a1e6-4963-af96-a241ec72238d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:04.949279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495cfff-a1e6-4963-af96-a241ec72238d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:04.984469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:04.985086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:04.985356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:04.985630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] PATH_INFO: `/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:04.987107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] RBAC: Authorizing `identity:get_role(role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:04.988474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:04.989790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 548/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:04 2022] GET /identity/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:04.997185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:05.124316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff51230-a6bc-4ef8-9d56-c96a32292df1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:05.131889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:05.162296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:05.162948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:05.163261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:05.163539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] PATH_INFO: `/v3/users/44d04e14decd4d14bca24ca2616ac723` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:05.180854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] RBAC: Authorizing `identity:get_user(user_id=44d04e14decd4d14bca24ca2616ac723)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:05.182324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:05.184039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 566/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:04 2022] GET /identity/v3/users/44d04e14decd4d14bca24ca2616ac723 => generated 286 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:05.190379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:05.257439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:05.314918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215478c7-4f82-4974-8507-88963cec3697 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:05.350287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:05.350908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:05.351222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:05.351496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:05.426408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:05.427922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:05.483637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:05.535093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 549/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:05.541460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:05.659249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e6798-1a26-45cb-9822-8ca71c2d9f47 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:05.700917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:05.701471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:05.701745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:05.702009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:05.703334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:05.704779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891649-f35d-48c0-93e0-e7f43af49724 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:05.706429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 567/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:05 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:05.712916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:05.824684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e82a4-f746-465e-8b39-bddfe6ef32e2 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:05.872299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:05.872894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:05.873168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:05.873435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:05.874182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=6a792c10e11846a0b16dd90126661829, user.id=44d04e14decd4d14bca24ca2616ac723) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:05.874485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:05.876003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce72835-8a41-46d9-a78a-1d1a67182f86 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:05.881508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 550/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:05 2022] GET /identity/v3/role_assignments?user.id=44d04e14decd4d14bca24ca2616ac723&scope.project.id=6a792c10e11846a0b16dd90126661829&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:07.744329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d39f3-3d14-4de1-8710-5e4b9331fcc8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:07.745209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d39f3-3d14-4de1-8710-5e4b9331fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:07.745209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d39f3-3d14-4de1-8710-5e4b9331fcc8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:07.747161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 568/1118] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:07.758887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:07.759661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:07.759815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:07.786873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:07.788090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:07.800521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:07.869977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 551/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:08.148438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d39f3-3d14-4de1-8710-5e4b9331fcc8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:08.185984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:08.263151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d39f3-3d14-4de1-8710-5e4b9331fcc8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:08.298662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:08.299325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:08.299599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:08.299849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] PATH_INFO: `/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:08.301314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] RBAC: Authorizing `identity:get_role(role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:08.302695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:08.304091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 569/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:08 2022] GET /identity/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:08.310425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:08.438156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1b690-9a4b-4d0e-a3c1-7e6fa42db54d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:08.474311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:08.474943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:08.475255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:08.475528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] PATH_INFO: `/v3/users/44d04e14decd4d14bca24ca2616ac723` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:08.476704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] RBAC: Authorizing `identity:get_user(user_id=44d04e14decd4d14bca24ca2616ac723)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:08.478132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:08.479744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 552/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:08 2022] GET /identity/v3/users/44d04e14decd4d14bca24ca2616ac723 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:08.486297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:08.589367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:08.593346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70b4f-53a2-423a-9285-ed6b53b664c4 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:08.638412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:08.639054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:08.639345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:08.639584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:08.640736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:08.642168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26b45-f49a-4488-a0a6-6929390a7e20 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:08.643839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 570/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:08 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:44:08.650650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:08.667436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:08.758724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddda1f-6b48-4ee3-84f9-ae8b1b282b3c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:08.794473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:08.795110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:08.795418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:08.795687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829/users/44d04e14decd4d14bca24ca2616ac723/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:08.797941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] RBAC: Authorizing `identity:create_grant(project_id=6a792c10e11846a0b16dd90126661829, user_id=44d04e14decd4d14bca24ca2616ac723, role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:08.799408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d1ca-96f0-4c84-9261-d3992263e4df None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:08.816140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 553/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:44:08 2022] PUT /identity/v3/projects/6a792c10e11846a0b16dd90126661829/users/44d04e14decd4d14bca24ca2616ac723/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:44:10.695341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf500b0a-e955-4b2f-8114-c7734b7d1bf8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:10.695892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf500b0a-e955-4b2f-8114-c7734b7d1bf8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:10.695892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf500b0a-e955-4b2f-8114-c7734b7d1bf8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:10.697145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 571/1124] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:10.703595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:10.704453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:10.704453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:10.729349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:10.730590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:10.803126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 554/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:11.081367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf500b0a-e955-4b2f-8114-c7734b7d1bf8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:11.202520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf500b0a-e955-4b2f-8114-c7734b7d1bf8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:11.240336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:11.241314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:11.241737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:11.242152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] PATH_INFO: `/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:11.244724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] RBAC: Authorizing `identity:get_role(role_id=00a5e081f3a643bc8083fc553ec1d6f5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:11.247260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:11.248741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 572/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:11 2022] GET /identity/v3/roles/00a5e081f3a643bc8083fc553ec1d6f5 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:11.254857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:11.258439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:11.371117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7a98e-8f18-4621-8e77-c641722ed859 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:11.409605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:11.410228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:11.410503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:11.410771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] PATH_INFO: `/v3/users/44d04e14decd4d14bca24ca2616ac723` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:11.411922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=44d04e14decd4d14bca24ca2616ac723)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:11.413932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:11.414937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 555/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:11 2022] GET /identity/v3/users/44d04e14decd4d14bca24ca2616ac723 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:11.421234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:11.458065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:11.521489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679f15f-6ab3-44a2-9ac6-76bd2cfeb948 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:11.560863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:11.561459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:11.561727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:11.561993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:11.627768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:11.629098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:11.663865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:11.733092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 573/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:11.739335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:11.822471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:11.856449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925ebd-f389-49d2-be0b-8f00133ee2a7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:11.891959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92303022-972b-4283-86f2-5527090527e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:11.892506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303022-972b-4283-86f2-5527090527e4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:11.892780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303022-972b-4283-86f2-5527090527e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:11.893049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92303022-972b-4283-86f2-5527090527e4 None admin] PATH_INFO: `/v3/projects/6a792c10e11846a0b16dd90126661829` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:11.894180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92303022-972b-4283-86f2-5527090527e4 None admin] RBAC: Authorizing `identity:get_project(project_id=6a792c10e11846a0b16dd90126661829)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:11.895659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92303022-972b-4283-86f2-5527090527e4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:11.897284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 556/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:11 2022] GET /identity/v3/projects/6a792c10e11846a0b16dd90126661829 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:11.903779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:12.020477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0e31-f81a-45d0-82b2-9bd1c29bdc2a None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:12.058878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:12.059494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:12.059768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:12.060034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:12.060748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] RBAC: Adding query filter params (role.id=00a5e081f3a643bc8083fc553ec1d6f5, scope.project.id=6a792c10e11846a0b16dd90126661829, user.id=44d04e14decd4d14bca24ca2616ac723) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:12.061055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:12.062518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3fddd-bf5a-4114-97c8-752450b58de4 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:12.065549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:12.068960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 574/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:11 2022] GET /identity/v3/role_assignments?user.id=44d04e14decd4d14bca24ca2616ac723&scope.project.id=6a792c10e11846a0b16dd90126661829&role.id=00a5e081f3a643bc8083fc553ec1d6f5 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:13.986839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ad64-e2f6-401a-b078-b3b40a10c6d2 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:13.987766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ad64-e2f6-401a-b078-b3b40a10c6d2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:13.987883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5ad64-e2f6-401a-b078-b3b40a10c6d2 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:13.989999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 557/1131] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:13.996706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:13.997645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:13.997645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:14.024182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:14.025407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:14.104986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 575/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:14.376592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ad64-e2f6-401a-b078-b3b40a10c6d2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:14.491441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5ad64-e2f6-401a-b078-b3b40a10c6d2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:14.528782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:14.529404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:14.529669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:14.529939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:14.531200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:14.532630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5dbe6-41d9-4b47-8c08-c8ed313f585a None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:14.534197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 558/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:44:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:14.540361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:14.648490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493532dc-cfea-4800-b9e8-bf865b54f9f2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:14.680209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:14.689029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:14.689950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:14.690379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:14.690806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:14.691878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:14.694178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2bcdd-7f7d-40a3-88d5-8abb81836c7e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:14.726555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 576/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:44:14 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:16.652903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d57c-13ae-4758-bcd0-58fceef4f457 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:16.653485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d57c-13ae-4758-bcd0-58fceef4f457 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:16.653485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250d57c-13ae-4758-bcd0-58fceef4f457 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:16.654800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 559/1135] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:16.661164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:16.661853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:16.662117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:16.689568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:16.691551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:16.765646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 577/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:17.038571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250d57c-13ae-4758-bcd0-58fceef4f457 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:17.051135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:17.155688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250d57c-13ae-4758-bcd0-58fceef4f457 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:17.191218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:17.191834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:17.192107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:17.192374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:17.193629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:17.195075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33bb85-b06f-4d2d-a50e-53d1fa3953e2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:17.196655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 560/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:44:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:17.203019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:17.316275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016951-b234-4349-b460-b30a8c03e2b7 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:17.330134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:17.357493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:17.358107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:17.358412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:17.358681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:17.359361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:17.360802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606b52-e157-4b0b-a654-0fbeb41833e0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:17.385339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 578/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:44:17 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:19.320894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf609a-8327-4c1e-9550-dba696cab717 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:19.321770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf609a-8327-4c1e-9550-dba696cab717 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:19.321770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf609a-8327-4c1e-9550-dba696cab717 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:19.324052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 561/1139] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:19.330218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:19.330662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:19.330770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:19.358984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:19.360211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:19.441763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 579/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:19.714637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf609a-8327-4c1e-9550-dba696cab717 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:19.810577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:19.849347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf609a-8327-4c1e-9550-dba696cab717 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:19.891590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:19.892212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:19.892485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:19.892754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:19.903990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:19.911728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:19.912995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:19.914123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 562/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:19.920108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:20.039515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:20.047136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4abd9-a399-4ce9-81f1-55b5fabcd2a3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:20.095028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:20.095659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:20.095937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:20.096204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:20.096866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:20.097168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:20.098545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:20.106864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 580/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:20.112927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:20.225448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c38cc7-bb00-4dc9-a7df-0c8752163fc1 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:20.261143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:20.261781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:20.262053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:20.262331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] PATH_INFO: `/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:20.265229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:20.276724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] RBAC: Authorizing `identity:get_user(user_id=fe46d2e40098434d8ee0f4b5085d6ebf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:20.278447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:20.280195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 563/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:20 2022] GET /identity/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:20.286553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:20.395072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c574-8cc8-4872-a38b-24de0d11242f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:20.432774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:20.433722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:20.434001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:20.434270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:20.462815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:20.508174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:20.509643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:20.615649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 581/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:20.622475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:20.735962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b222-a833-45c1-80ab-da5f8b556812 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:20.772949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:20.773569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:20.773843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:20.774110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] PATH_INFO: `/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:20.775281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] RBAC: Authorizing `identity:get_project(project_id=6b042c86d93a46fea9ca3f2770fd55cf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:20.776733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56cab5-280c-483a-a655-0b33af7e5948 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:20.778426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 564/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:20 2022] GET /identity/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:20.785025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:20.862821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:20.920890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9bb9d-33d0-47a5-b7af-62dc25a9fc86 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:20.959632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:20.960232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:20.960508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:20.960783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:20.961503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=6b042c86d93a46fea9ca3f2770fd55cf, user.id=fe46d2e40098434d8ee0f4b5085d6ebf) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:20.961808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:20.963270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84c9a-6edc-4808-9682-55220dde898e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:20.968996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 582/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:20 2022] GET /identity/v3/role_assignments?user.id=fe46d2e40098434d8ee0f4b5085d6ebf&scope.project.id=6b042c86d93a46fea9ca3f2770fd55cf&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:22.852483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b211d5-2a14-460e-a885-c58dd4ffdd9f None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:22.853030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b211d5-2a14-460e-a885-c58dd4ffdd9f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:22.853030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b211d5-2a14-460e-a885-c58dd4ffdd9f None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:22.855168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 565/1147] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:22.861292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:22.861720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:22.861856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:22.889041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:22.890238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:22.970730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 583/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:23.235723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b211d5-2a14-460e-a885-c58dd4ffdd9f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:23.318443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:23.370574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b211d5-2a14-460e-a885-c58dd4ffdd9f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:23.409113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:23.409741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:23.410023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:23.410289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:23.421048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:23.425734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:23.426777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:23.427761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 566/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:23.434230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:23.504477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:23.556964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af23fd9-ee58-4f36-bd3f-3221baaa4638 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:23.595772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:23.596370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:23.596646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:23.596912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:23.597575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:23.597874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:23.599281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:23.607481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 584/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:23.613713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:23.722242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17d05-bfaa-4636-93ed-0c8a55121f15 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:23.745030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:23.765755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:23.766335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:23.766588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:23.766834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] PATH_INFO: `/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:23.768218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=fe46d2e40098434d8ee0f4b5085d6ebf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:23.769648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:23.771244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 567/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:23 2022] GET /identity/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:23.777490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:23.887805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1026da-a51b-461f-bb35-1739c44f2c45 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:23.904357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:23.923362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:23.923975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:23.924248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:23.924516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] PATH_INFO: `/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:23.925751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6b042c86d93a46fea9ca3f2770fd55cf)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:23.927212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9e564-203e-4bf1-8a0c-98b5d70c75b2 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:23.928863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 585/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:23 2022] GET /identity/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:23.936280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:24.045426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07d49-f4b5-4040-b48d-979b4495d0e9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:24.081886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:24.082529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:24.082804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:24.083108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] PATH_INFO: `/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf/users/fe46d2e40098434d8ee0f4b5085d6ebf/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:24.085079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b042c86d93a46fea9ca3f2770fd55cf, user_id=fe46d2e40098434d8ee0f4b5085d6ebf, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:24.086505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f64411-10ac-431a-956a-a5891f375c77 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:24.107979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 568/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:44:23 2022] PUT /identity/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf/users/fe46d2e40098434d8ee0f4b5085d6ebf/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:44:25.988016 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9764-18c9-4e73-9dc7-1f95c5a36977 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:25.988906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9764-18c9-4e73-9dc7-1f95c5a36977 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:25.988906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9764-18c9-4e73-9dc7-1f95c5a36977 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:25.991156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 586/1154] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:25.999266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:25.999754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:25.999754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:26.019650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:26.020832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:26.068034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:26.090859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 569/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:44:26.355686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9764-18c9-4e73-9dc7-1f95c5a36977 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:26.473338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9764-18c9-4e73-9dc7-1f95c5a36977 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:26.510324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:26.510970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:26.511371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:26.511635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:26.522303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:26.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:26.528270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:44:26.529120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 587/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:26.536790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:26.649099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20c447-716d-49c7-bd3e-1af1f97dfd4e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:26.685254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:26.685861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:26.686134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:26.686400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:26.687056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:26.687374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:26.688715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:26.699283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 570/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:26.706314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:26.757220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:26.816540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a869-24f9-40f1-a078-3393354f1142 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:26.856433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:26.857055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:26.857327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:26.857593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] PATH_INFO: `/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:26.858698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] RBAC: Authorizing `identity:get_user(user_id=fe46d2e40098434d8ee0f4b5085d6ebf)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:26.860172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:26.861759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 588/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:26 2022] GET /identity/v3/users/fe46d2e40098434d8ee0f4b5085d6ebf => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:26.868546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:26.923262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:26.991893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37ae3-357d-429f-be95-b5ba19af64bd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:27.032445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:27.033188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:27.033282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:27.033542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:27.111279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:27.112802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:27.154620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:27.218638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 571/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:27.228145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:27.339307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee91d-65cc-4fb1-af8a-a5e8522f0e74 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:27.354546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:27.390061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:27.390688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:27.390962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:27.391405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] PATH_INFO: `/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:27.392622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] RBAC: Authorizing `identity:get_project(project_id=6b042c86d93a46fea9ca3f2770fd55cf)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:27.394090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742efcce-8378-4ac0-bed7-2b06dfd4e43b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:27.395805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 589/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:27 2022] GET /identity/v3/projects/6b042c86d93a46fea9ca3f2770fd55cf => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:27.402581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:27.509504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7224ec-3f6d-4733-b688-0685453f96ef None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:27.544508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:27.545111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:27.545382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:27.545657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:27.546369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=6b042c86d93a46fea9ca3f2770fd55cf, user.id=fe46d2e40098434d8ee0f4b5085d6ebf) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:27.546677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:27.548142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28605ffa-e40a-4185-8fd6-831202f26ce5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:27.554359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 572/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:27 2022] GET /identity/v3/role_assignments?user.id=fe46d2e40098434d8ee0f4b5085d6ebf&scope.project.id=6b042c86d93a46fea9ca3f2770fd55cf&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:29.465389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c946b4-1985-4f04-8703-77724bc26aaf None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:29.466277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c946b4-1985-4f04-8703-77724bc26aaf None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:29.466277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c946b4-1985-4f04-8703-77724bc26aaf None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:29.468449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 590/1162] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:29.477251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:29.477943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:29.478149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:29.498395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:29.504165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:29.505383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:29.573525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 573/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:29.841788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c946b4-1985-4f04-8703-77724bc26aaf None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:29.961825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c946b4-1985-4f04-8703-77724bc26aaf None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:29.997981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:29.998586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:29.998864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:29.999166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:30.059211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:30.082923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:30.084427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:30.210215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 591/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:30.218166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:30.321531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3237a6-fa01-44cc-a084-277cad884400 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:30.357380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:30.357988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:30.358265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:30.358536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:30.361757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:30.366897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:30.368810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2bed5784-b404-4b4b-9e1d-70e5379e560e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:30.381313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:30.382508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:44:30.383615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:30.387194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 574/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:44:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:30.388688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:30.491112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:30.510460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a1cf7-2a22-4bf5-9f79-2bfc3ca72864 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:30.545470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:30.546065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:30.546338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:30.546606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:30.547265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:30.547560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:30.548922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d534780-dc10-4fd9-8f96-3931e4c65d0d None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:30.556644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 592/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 30 00:44:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:32.492352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7470-47f4-4cfa-b57d-7bd9c71a584d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:32.492897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7470-47f4-4cfa-b57d-7bd9c71a584d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:32.492897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e7470-47f4-4cfa-b57d-7bd9c71a584d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:32.494270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 575/1167] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:32.501110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191da47-cf04-4ea3-83a5-6a2f3f1f7bd4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:32.501827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191da47-cf04-4ea3-83a5-6a2f3f1f7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:32.502035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191da47-cf04-4ea3-83a5-6a2f3f1f7bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:32.530540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1191da47-cf04-4ea3-83a5-6a2f3f1f7bd4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:32.531873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1191da47-cf04-4ea3-83a5-6a2f3f1f7bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:32.605006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 593/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:32.869306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e7470-47f4-4cfa-b57d-7bd9c71a584d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:32.975040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e7470-47f4-4cfa-b57d-7bd9c71a584d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:33.010471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:33.011174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:33.011450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:33.011718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:33.012363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:33.013726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4bdc-22ff-4c9d-9469-032586e6cb80 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:33.033963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 576/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Tue Aug 30 00:44:32 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:34.938083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d878ff-6db0-4abe-8c40-cf5baeda6c70 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:34.938992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d878ff-6db0-4abe-8c40-cf5baeda6c70 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:34.938992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d878ff-6db0-4abe-8c40-cf5baeda6c70 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:34.941221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 594/1170] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:34.947496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:34.948881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:34.948881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:34.968992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:34.970198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:35.028559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:35.047687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 577/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:35.341660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d878ff-6db0-4abe-8c40-cf5baeda6c70 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:35.436139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:35.464289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d878ff-6db0-4abe-8c40-cf5baeda6c70 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:35.501416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:35.502052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:35.502333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:35.502605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] PATH_INFO: `/v3/domains/ef44085a91904b13ba6681fe69d6f6d0` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:35.519573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef44085a91904b13ba6681fe69d6f6d0)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:35.521938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdbaad-2342-4775-b84e-ba0f759d7507 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:35.524506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 595/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Aug 30 00:44:35 2022] GET /identity/v3/domains/ef44085a91904b13ba6681fe69d6f6d0 => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:35.534628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:35.652166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d6320-4f32-4ca0-8f57-bd76e55befd6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:35.690694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:35.691327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:35.691610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:35.691888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:35.692529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:35.693970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129c3b7-f678-4f33-bafb-9916b9d8e08c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:35.726961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 578/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:44:35 2022] POST /identity/v3/projects => generated 361 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:37.681268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68412-4aa6-4200-a550-5c38add1027f None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:37.681854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68412-4aa6-4200-a550-5c38add1027f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:37.681854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68412-4aa6-4200-a550-5c38add1027f None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:37.683341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 596/1174] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:37.689704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:37.690450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:37.690593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:37.718570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:37.719812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:37.752066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:37.802862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 579/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:38.078087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68412-4aa6-4200-a550-5c38add1027f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:38.193709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68412-4aa6-4200-a550-5c38add1027f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:38.231984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:38.232605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:38.232876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:38.233143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] PATH_INFO: `/v3/domains/ef44085a91904b13ba6681fe69d6f6d0` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:38.234369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef44085a91904b13ba6681fe69d6f6d0)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:38.235840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b5c9-d990-4850-97cd-ba7b955bc08b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:38.237404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 597/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Aug 30 00:44:38 2022] GET /identity/v3/domains/ef44085a91904b13ba6681fe69d6f6d0 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:38.246330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:38.357327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715895b3-ca3b-481d-8d00-3d8d7dc84de7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:38.404008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:38.404618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:38.404891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:38.405160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:38.405817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:38.407278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37f172-4d67-4b82-883c-50aeffdc9764 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:38.432744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 580/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:44:38 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:44:40.362667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0edb9-0e5e-4194-b57e-8c24e83fc441 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:40.363304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0edb9-0e5e-4194-b57e-8c24e83fc441 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:40.363304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0edb9-0e5e-4194-b57e-8c24e83fc441 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:40.364579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 598/1178] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:40.370894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:40.371639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:40.371927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:40.397272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:40.398471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:40.423223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:40.486978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 581/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:40.761855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0edb9-0e5e-4194-b57e-8c24e83fc441 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:40.765176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:40.876243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0edb9-0e5e-4194-b57e-8c24e83fc441 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:40.921845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:40.922470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:40.922749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:40.923042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:40.933947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:40.938775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:40.939927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:44:40.941171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 599/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:40.946388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:41.065571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156b0dc-6473-49a2-99c0-8e92a9a72fcc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:41.101908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:41.102513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:41.102793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:41.103099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:41.103759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:41.104061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:41.105485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:41.114260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 582/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:41.120413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:41.191776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:41.242641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe8c78-4124-4162-88e9-21632d58b9c6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:41.281885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:41.282511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:41.282786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:41.283065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] PATH_INFO: `/v3/users/c6f77aff084b46e68f571db60b80ee27` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:41.297090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] RBAC: Authorizing `identity:get_user(user_id=c6f77aff084b46e68f571db60b80ee27)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:41.298544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:41.300190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 600/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:41 2022] GET /identity/v3/users/c6f77aff084b46e68f571db60b80ee27 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:41.310015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:41.332122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:41.426879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a65f7-2bda-4280-b429-7e78bbb81e0c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:41.465681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:41.466278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:41.466552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:41.466822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:41.535448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:41.536906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:41.553793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:41.641507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 583/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:41.648405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:41.763737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25703-6556-40c0-88d3-1003ec84dc88 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:41.784630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:41.802872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:41.803541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:41.803819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:41.804091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] PATH_INFO: `/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:41.805348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] RBAC: Authorizing `identity:get_project(project_id=e60ec550ac1a42e2aa4107c46c2b1d79)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:41.806792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994e2b6-b199-4226-ae10-ae0653f962a8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:41.808455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 601/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:41 2022] GET /identity/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:41.814905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:41.916036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a884d-87c8-43f6-bf60-ef32591bb0c7 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:41.932493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:41.952550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:41.953141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:41.953411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:41.953678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:41.954384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=e60ec550ac1a42e2aa4107c46c2b1d79, user.id=c6f77aff084b46e68f571db60b80ee27) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:41.954687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:41.956130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5b82b-6279-4eb5-83db-71a52bd117ac None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:41.961386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 584/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:41 2022] GET /identity/v3/role_assignments?user.id=c6f77aff084b46e68f571db60b80ee27&scope.project.id=e60ec550ac1a42e2aa4107c46c2b1d79&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:43.879625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a9e95-019a-463f-8da4-a9839aebcd36 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:43.880209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a9e95-019a-463f-8da4-a9839aebcd36 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:43.880209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a9e95-019a-463f-8da4-a9839aebcd36 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:43.881548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 602/1186] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:43.888302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:43.889036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:43.889184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:43.919377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:43.920588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:43.992088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 585/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:44.276617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a9e95-019a-463f-8da4-a9839aebcd36 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:44.390981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a9e95-019a-463f-8da4-a9839aebcd36 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:44.431139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:44.431771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:44.432042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:44.432305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:44.442372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:44.447085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:44.448236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:44:44.449136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 603/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:44.454457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:44.564339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b80c4-3e95-4ffb-9407-8eeeedbb94b1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:44.574952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:44.599934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:44.600537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:44.600812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:44.601108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:44.601765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:44.602062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:44.603505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:44.611526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 586/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:44.618184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:44.677069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:44.743458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ecf5-a558-4036-87c1-e6f3f2279b86 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:44.783637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:44.784263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:44.784538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:44.784810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] PATH_INFO: `/v3/users/c6f77aff084b46e68f571db60b80ee27` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:44.786047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] RBAC: Authorizing `identity:get_user(user_id=c6f77aff084b46e68f571db60b80ee27)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:44.787536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:44.789131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 604/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:44 2022] GET /identity/v3/users/c6f77aff084b46e68f571db60b80ee27 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:44.795642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:44.904295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c25c3-ff54-4d42-a335-d1e10be7e65c None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:44.939412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:44.940036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:44.940310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:44.940583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] PATH_INFO: `/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:44.941694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] RBAC: Authorizing `identity:get_project(project_id=e60ec550ac1a42e2aa4107c46c2b1d79)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:44.943149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd307e9-5694-4b44-b8e3-77cb73917a8b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:44.944756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 587/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:44 2022] GET /identity/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:44.951577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:45.067496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dfae-0d99-4fe8-8381-d03a4b4d4732 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:45.084613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:45.103826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:45.104478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:45.104774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:45.105054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] PATH_INFO: `/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79/users/c6f77aff084b46e68f571db60b80ee27/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:45.106946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ec550ac1a42e2aa4107c46c2b1d79, user_id=c6f77aff084b46e68f571db60b80ee27, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:45.108475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0fcea-368b-47b3-9a2b-ac801012ddee None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:45.123754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 605/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:44:44 2022] PUT /identity/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79/users/c6f77aff084b46e68f571db60b80ee27/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:44:47.009354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe3ac1-9e11-4c42-9ac9-df5ca3229357 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:47.009938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe3ac1-9e11-4c42-9ac9-df5ca3229357 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:47.009938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe3ac1-9e11-4c42-9ac9-df5ca3229357 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:47.011520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 588/1193] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:44:47.016906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:47.017381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:47.017486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:47.039086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:44:47.040320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:44:47.115856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 606/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:44:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:44:47.384716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe3ac1-9e11-4c42-9ac9-df5ca3229357 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:47.439847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:47.505487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe3ac1-9e11-4c42-9ac9-df5ca3229357 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:47.540531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:47.541155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:47.541428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:47.541697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:47.552474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:44:47.557153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:44:47.558454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:44:47.559874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 589/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:44:47.568225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:47.681454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd7ed9-718c-41ab-b4aa-b8d279e33435 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:47.689432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:47.722206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:47.723271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:47.723589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:47.723918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:47.724980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:47.725407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:47.727625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:47.737260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 607/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:47.746028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:47.849295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:47.852765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b2acc-54ab-4983-aeee-cedb6c9c28d0 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:47.888120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:47.888731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:47.889003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:47.889274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] PATH_INFO: `/v3/users/c6f77aff084b46e68f571db60b80ee27` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:47.890508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] RBAC: Authorizing `identity:get_user(user_id=c6f77aff084b46e68f571db60b80ee27)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:47.892002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:47.893565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 590/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 30 00:44:47 2022] GET /identity/v3/users/c6f77aff084b46e68f571db60b80ee27 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:44:47.900514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:48.022866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d407-4555-42b0-8788-7161ac507134 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:48.059127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:48.059722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:48.059999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:48.060268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:48.096383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:48.138508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:48.139997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:48.240931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 608/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:44:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:44:48.247605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:48.363638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2036d-600b-4797-8057-388910aa2423 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:48.398819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:48.399501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:48.399772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:48.400045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] PATH_INFO: `/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:48.401300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] RBAC: Authorizing `identity:get_project(project_id=e60ec550ac1a42e2aa4107c46c2b1d79)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:48.403040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b6824-147c-416f-9517-a6b77693a512 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:48.404680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 591/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 30 00:44:48 2022] GET /identity/v3/projects/e60ec550ac1a42e2aa4107c46c2b1d79 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:48.411351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:44:48.489442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:48.538054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ad602-c0ee-4177-803f-e11ee138e5c6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:44:48.572556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:44:48.573172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:44:48.573520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:44:48.573793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:44:48.574600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=e60ec550ac1a42e2aa4107c46c2b1d79, user.id=c6f77aff084b46e68f571db60b80ee27) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:44:48.574910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:44:48.576392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd84076-f926-4029-9ff4-273c6f899e63 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:44:48.582839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 609/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:44:48 2022] GET /identity/v3/role_assignments?user.id=c6f77aff084b46e68f571db60b80ee27&scope.project.id=e60ec550ac1a42e2aa4107c46c2b1d79&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:44:58.394158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=82 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:44:58.579697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=34 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:45.442087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100fa7-078c-43f0-8c5c-2e8d09bf6956 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:45.442718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100fa7-078c-43f0-8c5c-2e8d09bf6956 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:45.442718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100fa7-078c-43f0-8c5c-2e8d09bf6956 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:45.444030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 592/1201] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:45:45.451045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:45.451045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:45.451045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:45.476852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:45:45.478269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:45:45.563544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 610/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:45:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:45:45.912335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100fa7-078c-43f0-8c5c-2e8d09bf6956 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:46.052302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100fa7-078c-43f0-8c5c-2e8d09bf6956 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:46.091426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:46.092037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:46.092324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:46.092590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:46.104212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:46.107024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:46.128412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:46.130076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:46.131423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:46.131423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:46.131423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:46.133346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 593/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:46.141323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:46.294947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae6359-7c16-4bd9-a1ff-48349f64d4af None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:46.345310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:46.355682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:46.356441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:46.356856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:46.357179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:46.357825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:46.358127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:46.359940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b704580-3788-429c-9d6d-f4e56e12ebd5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:46.379138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 611/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:46.382539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:46.454371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:46.511654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae52be1-3935-4a64-aab4-00c46e975bbc None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:46.554665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:46.555307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:46.555739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:46.556121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:46.556965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:46.558742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38015dbb-2a1f-4bef-88ff-c48ec38064e1 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:46.599546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 594/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 30 00:45:46 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:45:48.691902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718ba-7b45-4dc0-b336-87ee47ccbe1c None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:48.692476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718ba-7b45-4dc0-b336-87ee47ccbe1c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:48.692476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718ba-7b45-4dc0-b336-87ee47ccbe1c None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:48.694839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 612/1206] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:45:48.699881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:48.700539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:48.700539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:48.721016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:45:48.722302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:45:48.803251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 595/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:45:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:45:49.076557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432718ba-7b45-4dc0-b336-87ee47ccbe1c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:49.225155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432718ba-7b45-4dc0-b336-87ee47ccbe1c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:49.269782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:49.270484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:49.270852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:49.271117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:49.285286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:45:49.290770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:49.291932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:49.296663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 613/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:45:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:45:49.298390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:49.343517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:49.411892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf43af-499b-4782-8b6a-a1d4e39f9a23 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:49.452599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:49.453226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:49.453502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:49.453773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:49.454464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:49.454765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:49.456215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:49.465816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 596/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:45:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:49.472796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:49.532017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:49.609206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a556-4a0a-46ef-aec5-e7a120765d92 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:49.649238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:49.649970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:49.650342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:49.650679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:49.662552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:49.664135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:49.681124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:49.682302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:49.683455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:49.683455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:49.683455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:49.684934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 614/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:49.691695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:49.826445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:49.829961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bada4-6b65-4ab5-9297-9cdf529c36f5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:49.871520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:49.872336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:49.872717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:49.872986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:49.873688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:49.874034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:49.875547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:49.891620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 597/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:45:49.900296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:50.023768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a5e1-9078-4b93-9205-5d63fcc879b7 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:50.025609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:50.062569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:50.063314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:50.063590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:50.063995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] PATH_INFO: `/v3/users/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:50.071831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:50.073359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:45:50.086493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:50.091233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:50.092491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 615/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:50.094811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:50.225576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ec13-0ae4-4303-bee9-066fcd1c80d3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:50.269611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:50.270679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:50.270679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:50.270905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:50.271695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:50.272007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:50.273712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:50.299045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 598/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:45:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:50.302646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:50.428102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e9ce0-69f4-4b22-b718-695c18a6a67b None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:50.473487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:50.474274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:50.474605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:50.474964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:50.513841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:50.568261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:50.570435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:50.704946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 616/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:45:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:45:50.706943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:50.729695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:50.829981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a35ebd-cf4e-4968-93e2-824b371a4379 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:50.882160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:50.883260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:50.883727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:50.884275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:50.900387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:50.902440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:50.925627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:50.928056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:50.929140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:50.929140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:50.929140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:50.929140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 599/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:50.934091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:51.046897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:51.117667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72c1c3-57e5-4acd-b3a4-42467ed82704 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:51.178830 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:51.179951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:51.180444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:51.180932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:51.182115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:51.182651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:51.185725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:51.204239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 617/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:51.210513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:51.319914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:51.381500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9085a3-9f2c-4182-96f0-e259c6823919 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:51.428755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:51.429437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:51.429715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:51.429986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:51.436818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:51.438299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:45:51.449600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:51.455987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:51.456963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 600/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:45:51.457505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:51.566380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f40c5f-83fe-469c-aeef-8774ca42c429 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:51.594644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:51.609198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:51.609945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:51.610230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:51.610599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:51.611457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:51.611769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:51.613428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878010eb-b3da-433c-b012-edeffa02d9ec None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:51.629676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 618/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:51.637166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:51.762540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885523b-2159-4aac-a34c-f474d22af2ae None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:51.794464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:51.802251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:51.802989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:51.803359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:51.803690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:51.804496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b22b14f894354d4197834889e75489a5) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:51.805037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:51.806579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae89098-7ab2-44bc-a250-f92dcf928e6f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:51.813533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 601/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:45:51 2022] GET /identity/v3/role_assignments?user.id=b22b14f894354d4197834889e75489a5&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:45:54.130254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e337-73cf-4859-9230-02e10565a25f None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:54.130876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e337-73cf-4859-9230-02e10565a25f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:54.130876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499e337-73cf-4859-9230-02e10565a25f None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:54.132469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 619/1220] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:45:54.140763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:54.141304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:54.141304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:54.161761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:45:54.162974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:45:54.243526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 602/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:45:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:45:54.543419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499e337-73cf-4859-9230-02e10565a25f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:54.656230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499e337-73cf-4859-9230-02e10565a25f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:54.692005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:54.692572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:54.692883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:54.693153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:54.704383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:45:54.709389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:54.710578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:54.711567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 620/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:45:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:45:54.717872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:54.831672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935125c-4e13-47f3-9117-4a8d7e6c2f54 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:54.860977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:54.876245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:54.876864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:54.877145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:54.877416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:54.878078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] RBAC: Adding query filter params (name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:54.878381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:54.879801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:54.889494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 603/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:54.895623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:54.946013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:55.012440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d1040-82b8-42b8-87ff-b1fdbe207c28 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.047157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:55.047792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:55.048069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:55.048338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:55.058744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:55.060527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:55.077524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:55.078592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:55.079530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:55.079530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:55.079530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:55.079530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 621/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:55.085915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:55.210539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e763dd2-6be8-48a5-9507-041431168364 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.249780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:55.250400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:55.250679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:55.250951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:55.251621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:55.251924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:55.253291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:55.268499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 604/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:55.275600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:55.388950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:55.397173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583f7a-fa14-484f-9dcf-3fc281b63729 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.438121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:55.438855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:55.439072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:55.439347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] PATH_INFO: `/v3/users/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:55.447826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:55.449653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:45:55.463752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:55.464998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:55.466161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 622/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:45:55.470192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:55.520921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:55.585267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cc516-04b5-4af9-af4b-2f799fd92aa3 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.627046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:55.627748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:55.628052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:55.628387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:55.629174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:55.629519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:55.631179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:55.650355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 605/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:45:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:55.657086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:55.776691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc971bd-2556-415d-ad18-cd585c7d9dde None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.820466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:55.821235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:55.821538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:55.821887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:55.834160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:55.835759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:55.856567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:55.857802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:55.858873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:55.858873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:55.858873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:55.858873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 623/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:55.864529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:55.984569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19235b9-688f-4f89-ad62-fe484c606608 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:55.987247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:56.024233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:56.024925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:56.025211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:56.025501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:56.026230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:56.026587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:56.028466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:56.045494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 606/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:45:56.046781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:56.057779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:56.169514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598977-2ca8-4166-9fbb-75a0885ba1c6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:56.204477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:56.205192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:56.205443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:56.205737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:56.214413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:56.216549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:45:56.229047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:56.230101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:45:56.231977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 624/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:56.236515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:56.348826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9b705-1de1-465e-8243-7949be249d54 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:56.388914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:56.389539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:56.389808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:56.390074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:56.390761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:56.391090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:56.392603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a67b8-36d3-4148-8479-4c928a6b6ff2 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:56.406083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 607/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:56.412375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:56.480944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:56.518375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee399c0-2827-4db3-bae0-ca65ae9992aa None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:56.556351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:56.557100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:56.557384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:56.557642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/b22b14f894354d4197834889e75489a5/roles/a641de721fec4e50b18e9fdafb825b38` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:56.574912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=b22b14f894354d4197834889e75489a5, role_id=a641de721fec4e50b18e9fdafb825b38)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:56.576423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee1ef4-5d35-4c7e-bd16-2e11458ac33a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:56.591818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 625/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:45:56 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/b22b14f894354d4197834889e75489a5/roles/a641de721fec4e50b18e9fdafb825b38 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:45:58.583825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2801ce-676e-4336-926c-d5e8c85b035f None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:58.584405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2801ce-676e-4336-926c-d5e8c85b035f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:58.584542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2801ce-676e-4336-926c-d5e8c85b035f None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:58.586190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 608/1233] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:45:58.596062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:58.596062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:58.596062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:58.614285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:45:58.615512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:45:58.682685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 626/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:45:58.973259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2801ce-676e-4336-926c-d5e8c85b035f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.002401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:59.092095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2801ce-676e-4336-926c-d5e8c85b035f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:59.131351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:59.132039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:59.132301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:59.132570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:59.146233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:45:59.150963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:59.152067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:59.153139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 609/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 30 00:45:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:59.158843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.212695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:59.269264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a8065-2826-44f1-8611-f6a01d6e5016 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:59.303129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:59.303746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:59.304017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:59.304282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:59.304967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] RBAC: Adding query filter params (name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:59.305231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:59.306602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:59.315148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 627/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 30 00:45:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:59.321774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.415640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:59.437398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241eb0-27be-4014-8fab-72b340ca5a5f None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:59.473495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:59.474172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:59.474488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:59.474731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:59.486740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:59.488377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:45:59.503960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:59.505017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:45:59.506113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:45:59.506113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:45:59.506113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:59.506113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 610/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:59.514054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.624316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a1a-b736-4c24-bc4b-7efaaf8edf1c None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:59.633981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:59.660394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:59.661012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:59.661309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:59.661575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:59.662270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:45:59.662556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:59.664019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:59.677186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 628/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:45:59.684871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.788263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6509f-1a81-4602-a7a0-87a8767ec152 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:45:59.825130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:45:59.825765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:45:59.826066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:45:59.826338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] PATH_INFO: `/v3/users/placement` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:45:59.829804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:45:59.836136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:45:59.838225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:45:59.851095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:45:59.852132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:45:59.853278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 611/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:45:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:45:59.859045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:45:59.986946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f300ae-f6d5-46c7-be94-600195712716 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:00.030689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:00.031517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:00.031792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:00.032064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:00.033045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:00.033333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:00.035288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:00.056577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 629/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:45:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:00.062882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:00.175309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bcd7-03f6-470f-8d55-caa135e7420d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:00.214267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:00.215143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:00.215426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:00.215700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:00.282279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:00.289760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:00.291678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:00.406370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 612/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:46:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:46:00.418345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:00.448764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:00.531410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4768b2-573d-4afd-b611-c2502b33e4c5 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:00.572892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:00.573593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:00.573896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:00.574166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:00.585230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:00.587247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:00.604920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:00.610784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:00.612595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:00.612595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:00.612595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:00.612595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 630/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:00.612886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:00.722783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:00.729451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75afd9b-af0b-4535-8d82-c4d8ad3697e5 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:00.765778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:00.766426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:00.766843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:00.767180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:00.767857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:00.768288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:00.769816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:00.782490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 613/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:46:00.788607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:00.890327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:00.916169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f832-765b-4eda-8c46-4431bc689dbd None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:00.954024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:00.954663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:00.954934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:00.955231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:00.962710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:00.964230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:46:00.975367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:00.976410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:00.977505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 631/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:00.982772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:01.093107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304632d-1ab4-44ba-bd51-8ea509b4d9a4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:01.129438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:01.130074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:01.130351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:01.130623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:01.131344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:01.131652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:01.133090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62c216-7c3b-4c82-a269-ba1a9427f556 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:01.147684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 614/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:46:01.155399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:01.276066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e8b55-bc94-4bdc-a1da-b1a77287ed25 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:01.320247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:01.320992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:01.321288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:01.321587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:01.322454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] RBAC: Adding query filter params (role.id=a641de721fec4e50b18e9fdafb825b38, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b22b14f894354d4197834889e75489a5) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:01.322789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:01.324544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dab6a9-7d94-4ca4-9407-4d6932567bef None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:01.331567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 632/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:46:01 2022] GET /identity/v3/role_assignments?user.id=b22b14f894354d4197834889e75489a5&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=a641de721fec4e50b18e9fdafb825b38 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:03.497107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab3e6-27a6-4bb4-b567-38c7efcc38c0 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:03.497668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab3e6-27a6-4bb4-b567-38c7efcc38c0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:03.497668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab3e6-27a6-4bb4-b567-38c7efcc38c0 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:03.499359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 615/1247] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:03.507613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:03.508191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:03.508191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:03.517797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:03.535265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:03.536827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:03.618286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 633/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:46:03.890483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ab3e6-27a6-4bb4-b567-38c7efcc38c0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:03.916793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:04.018342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ab3e6-27a6-4bb4-b567-38c7efcc38c0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:04.057735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:04.058382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:04.058668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:04.059031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:04.070441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:46:04.075447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:04.076483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:04.077757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 616/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:04.082914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:04.194951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2efa-3fb5-429e-8608-f6f7c36f3f96 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:04.230009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:04.237307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:04.238116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:04.238462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:04.238748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:04.239540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:04.239981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:04.241578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:04.252878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 634/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:04.259066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:04.347733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:04.374142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b963c4-b3ea-4d3c-a1fa-03c18c0e57e4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:04.419415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:04.420525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:04.420976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:04.421416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:04.438361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:04.440299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:04.467194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:04.468345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:04.469304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:04.469304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:04.469304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:04.469304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 617/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:46:04.477579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:04.597045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40f12-8ce8-44c7-a066-c860ae3b9f51 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:04.634974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:04.635749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:04.636256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:04.636691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:04.637460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:04.637874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:04.639356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:04.651708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 635/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:04.657816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:04.768823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f910a5-3878-4a96-be5f-59a4411a972b None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:04.797795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:04.808769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:04.809490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:04.809838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:04.810180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:04.818071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:04.819542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:46:04.831439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:04.832621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:04.833800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 618/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:04.839107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:04.910177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:04.965156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232fb3-4d56-4303-b8e4-945416ee490f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:05.003244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:05.003872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:05.004119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:05.004384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:05.005132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:05.005412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:05.006853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:05.026699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 636/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:46:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:46:05.032659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:05.143345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da02267-1997-4151-a704-aec975afb8fd None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:05.180657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:05.181298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:05.181546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:05.181813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:05.242709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:05.259840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:05.261656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:05.372268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 619/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:46:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:46:05.378837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:05.490617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2f893-3422-42cd-928d-95cd85033706 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:05.507279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:05.533553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:05.534185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:05.534490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:05.534720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:05.545696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:05.547463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:05.564688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:05.565878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:05.566799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:05.566799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:05.566799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:05.566799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 637/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:05.571876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:05.673953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:05.686928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a95f9-25a5-4521-95d2-04a4f9e076db None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:05.725588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:05.726205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:05.726484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:05.726760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:05.727460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:05.727745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:05.729130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:05.742191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 620/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:05.749627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:05.861563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981c3a6-bfb0-4e69-9fdb-40db54631c7e None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:05.901765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:05.902465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:05.902822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:05.903076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:05.910819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:05.912454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:46:05.925386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:05.926464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:05.927546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 638/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:05.932959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:06.044128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c9e5-7cbc-4513-b3d0-9a3aa2d05ee6 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:06.082427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:06.083133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:06.083441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:06.083683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:06.084370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:06.084668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:06.086159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea86d8-f9b0-43b6-b929-67f7fd50879f None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:06.099815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 621/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:06.107162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:06.136046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:06.224139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5631f-44bf-4ee6-abe4-5c44ff8714e8 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:06.261145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:06.261721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:06.261996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:06.262269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:06.262999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b22b14f894354d4197834889e75489a5) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:06.263381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:06.264832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485968-9f76-4d61-b724-f8670c8af22a None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:06.271191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 639/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:46:06 2022] GET /identity/v3/role_assignments?user.id=b22b14f894354d4197834889e75489a5&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:08.181093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbae21-bd9e-448d-9d76-52662b5fb514 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:08.181668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbae21-bd9e-448d-9d76-52662b5fb514 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:08.181668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbae21-bd9e-448d-9d76-52662b5fb514 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:08.183235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 622/1261] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:08.188769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:08.189225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:08.189316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:08.215929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:08.217163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:08.295840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 640/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:46:08.563953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbae21-bd9e-448d-9d76-52662b5fb514 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:08.579400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:08.676971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbae21-bd9e-448d-9d76-52662b5fb514 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:08.715473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:08.716095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:08.716370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:08.716632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:08.727577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:46:08.732381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:08.733507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:08.734828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 623/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:08.740364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:08.769007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:08.852561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7333-69f2-4413-a8a2-e96a46a7e235 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:08.898829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:08.899597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:08.899939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:08.900292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:08.901107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:08.901578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:08.903766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:08.913934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 641/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:08.920582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.006716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:09.044518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f4e-cc25-4c9d-91b5-cbbe1a4ca588 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.083342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.083971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.084249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.084520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.095586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:09.097041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:09.112359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:09.113441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:09.114467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:09.114467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:09.114467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:09.114467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 624/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:09.119581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.204643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:09.233562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022cd2-54cf-453d-b544-a512107f0d07 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.269653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.270576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.271056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.271474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.272497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:09.272980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:09.275241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:09.293772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 642/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:09.299816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.414026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd476960-0566-4e56-b26b-a40eaa4898c4 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.436208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:09.451661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.452308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.452568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.452837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] PATH_INFO: `/v3/users/placement` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.459812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:09.461246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:46:09.473209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:09.478740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:09.480435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 625/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:09.481202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.583952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffcfdf-10a1-491e-a742-acd8ee4ebf1f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.622591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.623218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.623487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.623754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.624479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:09.624776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:09.626205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:09.635519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:09.645380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 643/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:46:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:09.652279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.768483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669216d-6082-4ef8-818b-da704c81d756 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.804489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.805107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.805381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.805649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.816858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:09.818311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:09.833557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:09.834634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:09.835863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:09.835863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:09.835863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:09.835863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 626/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:09.844616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:09.960863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b18f94-e71c-48e2-a779-39d98fc2448f None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:09.997484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:09.998081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:09.998360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:09.998616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:09.999285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:09.999589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:10.000958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:10.017169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 644/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:10.023653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:10.056860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:10.127627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4e2a5-8b78-4695-a4e4-3b72d42da247 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:10.160845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:10.161473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:10.161748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:10.162021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:10.169377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:10.170825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:46:10.182259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:10.183389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:10.184550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 627/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:10.190272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:10.256643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:10.298464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2e6a7-8b67-485d-9bdc-0499b4f2f1a0 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:10.332292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:10.332888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:10.333162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:10.333426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:10.334100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:10.334412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:10.335878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cab8e-1a61-49f0-a4b9-b48f1671c3cf None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:10.348526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 645/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:10.355056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:10.458387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:10.467945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490517fe-24ff-42ad-a114-85bc22725cb9 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:10.502801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:10.503462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:10.503735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:10.504009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] PATH_INFO: `/v3/projects/920fb02470584601a6dfea23df009f85/users/b22b14f894354d4197834889e75489a5/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:10.505909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] RBAC: Authorizing `identity:create_grant(project_id=920fb02470584601a6dfea23df009f85, user_id=b22b14f894354d4197834889e75489a5, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:10.507382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ab0e0-be5c-41f5-a57f-d37ae4d9994c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:10.522996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 628/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 30 00:46:10 2022] PUT /identity/v3/projects/920fb02470584601a6dfea23df009f85/users/b22b14f894354d4197834889e75489a5/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:46:12.452518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc0fb-abc8-4f96-8eb2-fd915a8c2cce None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:12.453084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc0fb-abc8-4f96-8eb2-fd915a8c2cce None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:12.453084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc0fb-abc8-4f96-8eb2-fd915a8c2cce None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:12.454432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 646/1274] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:12.461063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:12.462200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:12.462412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:12.487978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:12.489182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:12.566497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 629/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:46:12.842877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc0fb-abc8-4f96-8eb2-fd915a8c2cce None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:12.961266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc0fb-abc8-4f96-8eb2-fd915a8c2cce None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:12.979836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:12.999732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.000365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.000641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.000911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.012400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:46:13.017230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:13.018293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:46:13.019292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:13.023599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 647/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 30 00:46:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:13.024927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:13.122785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:13.143346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608fe3d-42d3-424f-95f8-ec4b076d992e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:13.178188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.178790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.179084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.179353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.179999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:13.180295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:13.181660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:13.189908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 630/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:13.196608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:13.302751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1152-7cd8-4eea-87c7-f004845898fc None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:13.342763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.343458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.343712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.343978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.354231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:13.355791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:13.370958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:13.372204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:13.373739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:13.373739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:13.373739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:13.373739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 648/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:13.381761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:13.495281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db98c-b26b-4d23-b6bd-2d89f962a331 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:13.530123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.530727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.531043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.531317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.531951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:13.532253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:13.533632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:13.545483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 631/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:13.551540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:13.588452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:13.666090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c716f-75a3-4cbc-a258-43d8f9ec1a61 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:13.709817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.710684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.710961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.711280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.723282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:13.724733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:46:13.736449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:13.737537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:46:13.738650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:13.742344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 649/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:46:13.743525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:13.747623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:13.845017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd21b-45bb-4a2d-993b-455c3859d69e None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:13.883968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:13.884526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:13.884834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:13.885108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:13.885992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:13.886278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:13.887826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:13.906436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 632/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:46:13.912421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:14.021114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:14.031748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa389a-7fe2-4f86-a46d-058f5ac2cb40 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:14.067242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:14.067846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:14.068119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:14.068388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:14.142419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:14.143925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:14.240269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:14.252023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 650/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 30 00:46:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:46:14.258656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:14.332577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:14.372880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b290a-a10d-45ba-93c3-40500a8148c8 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:14.420751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:14.421436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:14.421721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:14.421992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:14.433253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:14.434718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:46:14.449942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:14.451028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:46:14.452263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:46:14.452263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:46:14.452263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:14.452263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 633/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:14.457812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:14.565943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c18e-7bc3-4553-856b-1068e13210c6 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:14.601790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:14.602399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:14.602670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:14.602938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:14.603616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:14.603923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:14.605487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:14.616767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 651/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 30 00:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:14.622838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:14.741498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de00e8-0c8e-4b61-b4f1-4de21c3e0874 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:14.755568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:14.778049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:14.778672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:14.778945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:14.779245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] PATH_INFO: `/v3/projects/service` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:14.786162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:14.787642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:46:14.798898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:14.800009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: ERROR keystone.server.flask.application Aug 30 00:46:14.802077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 634/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 30 00:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:14.809741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:14.862805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:14.924982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc98c98-8ea9-4b2d-aaf8-a875199d93d3 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:14.959692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:14.960286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:14.960562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:14.960825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:14.961872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:14.962170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:14.963640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7fdc5-8455-4bfb-94cb-87aa68b0ae61 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:14.977652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 652/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 30 00:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:14.984031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:15.126368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57a060-b9d3-4326-b461-d4fc578c437d None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:15.165771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:15.166370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:15.166643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:15.166933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:15.167687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] RBAC: Adding query filter params (role.id=e9f15fd6c7df4864ae7b25f72b495b90, scope.project.id=920fb02470584601a6dfea23df009f85, user.id=b22b14f894354d4197834889e75489a5) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:15.167989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:15.169408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854d3a1-2b9f-442d-97c5-5be15f7da0fb None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:15.175231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 635/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 30 00:46:14 2022] GET /identity/v3/role_assignments?user.id=b22b14f894354d4197834889e75489a5&scope.project.id=920fb02470584601a6dfea23df009f85&role.id=e9f15fd6c7df4864ae7b25f72b495b90 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:17.060386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499f58-a18b-4c71-9bd8-4f04651464ec None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:17.061255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499f58-a18b-4c71-9bd8-4f04651464ec None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:17.061255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499f58-a18b-4c71-9bd8-4f04651464ec None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:17.063497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 653/1288] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:17.072078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:17.072734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:17.072987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:17.099121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:17.100310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:17.132505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:17.172675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 636/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:46:17.437990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499f58-a18b-4c71-9bd8-4f04651464ec None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:17.553090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499f58-a18b-4c71-9bd8-4f04651464ec None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:17.564581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:17.596207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:17.597135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:17.597541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:17.597933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] PATH_INFO: `/v3/services/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:17.598837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:17.600990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:17.615227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:17.616843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:17.617821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 654/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:46:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:17.624253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:17.727720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab18fe-738e-449c-8649-b0f38e5f56d5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:17.765603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:17.766204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:17.766480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:17.766750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:17.767402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] RBAC: Adding query filter params (name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:17.767706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:17.769076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e47b82-29ba-4844-a285-8739edc1c75c None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:17.774666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 637/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 30 00:46:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:17.781765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:17.914067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a432a6-17e4-4fbe-83ee-d9583e40f774 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:17.952857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:17.953472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:17.953752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:17.954024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] PATH_INFO: `/v3/services` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:17.954660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] RBAC: Adding query filter params (type=placement) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:17.954966 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:17.956407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f147-5e86-49a7-b4b4-c97056f2ac3b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:17.960825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 655/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 30 00:46:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:19.847832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07daf70-eef8-4685-a7b9-49302f7e2068 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:19.848408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07daf70-eef8-4685-a7b9-49302f7e2068 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:19.848408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07daf70-eef8-4685-a7b9-49302f7e2068 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:19.849799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 638/1293] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:19.856612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2ff7-8556-4fba-ab37-57c712ce3dd1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:19.857464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2ff7-8556-4fba-ab37-57c712ce3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:19.857669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2ff7-8556-4fba-ab37-57c712ce3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:19.890725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4a6d2ff7-8556-4fba-ab37-57c712ce3dd1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:19.892568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4a6d2ff7-8556-4fba-ab37-57c712ce3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:19.944353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:19.993509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 656/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:46:20.271213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07daf70-eef8-4685-a7b9-49302f7e2068 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:20.319783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:20.401855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07daf70-eef8-4685-a7b9-49302f7e2068 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:20.446966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:20.447615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:20.447892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:20.448163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:20.448894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:20.450276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87cd67-a45b-445f-9b23-dfe511f23b40 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:20.466589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 639/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 30 00:46:20 2022] POST /identity/v3/services => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:46:22.359218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af492e8-3934-4757-bca0-8318a026619e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:22.360146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af492e8-3934-4757-bca0-8318a026619e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:22.360146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af492e8-3934-4757-bca0-8318a026619e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:22.362275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 657/1296] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:22.370171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:22.371107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:22.371107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:22.390836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:22.392076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:22.475750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 640/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:46:22.739388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af492e8-3934-4757-bca0-8318a026619e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:22.856994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af492e8-3934-4757-bca0-8318a026619e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:22.899827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:22.900642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:22.901004 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:22.901302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] PATH_INFO: `/v3/services/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:22.902150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:22.903947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:22.913806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:22.919371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:22.920305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 658/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:46:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:22.922804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:22.945680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:23.026743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a9196-1fc9-4279-a63d-ed6c2f5a662b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:23.060918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:23.061531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:23.061807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:23.062071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:23.062690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] RBAC: Adding query filter params (name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:23.062987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:23.064453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:23.069464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 641/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 30 00:46:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:23.079320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:23.116912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:23.183964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26316c41-d488-47db-b8fe-714fc390d889 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:23.218330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:23.218922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:23.219223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:23.219490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:23.220178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] RBAC: Adding query filter params (interface=public, service_id=51b101059b644587a68dcdb110a42a6a, region_id=RegionOne) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:23.220481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:23.221885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa059e0-cddd-4e53-a5f3-4d00167f3f7b None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:23.227064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 659/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 30 00:46:23 2022] GET /identity/v3/endpoints?service_id=51b101059b644587a68dcdb110a42a6a&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:23.233660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:23.339595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:23.358973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1f0ca-3ab0-4386-b8fa-ce8f4dc23f63 None admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:23.394934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:23.395608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:23.395850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:23.396114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:23.396740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:23.398103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5040-add5-4fb0-86d8-2e6442f20910 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:23.409514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 642/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 30 00:46:23 2022] GET /identity/v3/services => generated 2191 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:46:25.340880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cec44-7067-4a99-9ee2-5ca060e9c363 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:25.341464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cec44-7067-4a99-9ee2-5ca060e9c363 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:25.341464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cec44-7067-4a99-9ee2-5ca060e9c363 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:25.343525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 660/1302] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:46:25.352166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:25.352567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:25.352755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:25.378909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:46:25.380142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:46:25.462761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 643/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:46:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:46:25.727201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cec44-7067-4a99-9ee2-5ca060e9c363 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:25.830565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:25.841869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cec44-7067-4a99-9ee2-5ca060e9c363 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:25.879428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:25.880066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:25.880346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:25.880613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] PATH_INFO: `/v3/services/placement` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:25.881202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:25.882595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:46:25.892210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:46:25.893658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application with Lock( Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: ERROR keystone.server.flask.application Aug 30 00:46:25.894723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 661/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 30 00:46:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:46:25.899953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:25.992587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:26.018670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ff3d9-f9c1-40d5-9be6-83ffd51f9ca3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:26.053989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:26.054594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:26.054875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:26.055175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] PATH_INFO: `/v3/services` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:26.055800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:46:26.056101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:26.057507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0428e86-426b-440a-a78f-8645a7c25ee6 None admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:26.063234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 644/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 30 00:46:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:46:26.069549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:46:26.186213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af9bd-bc3b-4f5b-b02f-2cb71b5aba49 None admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:46:26.222235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:46:26.222834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:46:26.223139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:46:26.223412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:46:26.224088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:46:26.225477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d77d7-bbb0-417c-b61b-e59bc2bca944 None admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:46:26.253161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 662/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:46:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:46:36.059727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=30 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:46:36.245412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:30.906310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972a25-4234-45a5-b5e0-f63e7f7676b5 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:30.906890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972a25-4234-45a5-b5e0-f63e7f7676b5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:30.906890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3972a25-4234-45a5-b5e0-f63e7f7676b5 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:30.912577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 645/1307] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:30.914491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf8b7-53fe-487c-b897-b4fa0f283bed None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:30.915471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf8b7-53fe-487c-b897-b4fa0f283bed None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:30.916106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaf8b7-53fe-487c-b897-b4fa0f283bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:31.002684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dddaf8b7-53fe-487c-b897-b4fa0f283bed None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:31.004196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dddaf8b7-53fe-487c-b897-b4fa0f283bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:31.143055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 663/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:47:31.499059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:31.499386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:31.499723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:31.501292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 646/1309] 10.222.0.17 () {54 vars in 965 bytes} [Tue Aug 30 00:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:31.509286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4002cd-ea4e-4986-9a6f-62d9d4d1d4b5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:31.510411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4002cd-ea4e-4986-9a6f-62d9d4d1d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:31.510964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4002cd-ea4e-4986-9a6f-62d9d4d1d4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:31.596530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1a4002cd-ea4e-4986-9a6f-62d9d4d1d4b5 None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:31.597946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1a4002cd-ea4e-4986-9a6f-62d9d4d1d4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:31.620580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:31.679670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 664/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Aug 30 00:47:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:31.688754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:31.690673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:31.815830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbf08c-bbe1-4e01-a795-c13f8136bb63 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:31.902584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:31.903577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:31.903960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:31.904332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:31.906229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:31.972377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:32.039964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:32.041965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33ec5a-9093-46f2-9328-4b73785be95e service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:32.061581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 647/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:34.237002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6289bc-3af4-4e68-8f3c-b4687c04bcff None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.237596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6289bc-3af4-4e68-8f3c-b4687c04bcff None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.237596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6289bc-3af4-4e68-8f3c-b4687c04bcff None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.238948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 665/1312] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:34.244960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b586b-6533-4bb6-a056-10ee3d263ef4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.245569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b586b-6533-4bb6-a056-10ee3d263ef4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.245569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b586b-6533-4bb6-a056-10ee3d263ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.339257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3c0b586b-6533-4bb6-a056-10ee3d263ef4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:34.340635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3c0b586b-6533-4bb6-a056-10ee3d263ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:34.444997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 648/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:34.752650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.753276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.753790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.755990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 666/1314] 10.222.0.17 () {54 vars in 965 bytes} [Tue Aug 30 00:47:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:34.763420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1a353-1001-40e3-9823-92164c9dfa8e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:34.763785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1a353-1001-40e3-9823-92164c9dfa8e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:34.764178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1a353-1001-40e3-9823-92164c9dfa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:34.780786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:34.850729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-82f1a353-1001-40e3-9823-92164c9dfa8e None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:34.851982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-82f1a353-1001-40e3-9823-92164c9dfa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:34.929800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 649/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Aug 30 00:47:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:34.944838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:34.947423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:35.063698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef69f4-eeed-4517-8f24-f34cd301c9b9 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:35.139039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:35.143796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:35.144420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:35.144708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:35.144977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:35.147363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:35.272339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:35.273795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46265d3b-2705-4058-8ecb-2c2d5c22b04d service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:35.292516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 667/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:37.773281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.774168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.774168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.776617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 650/1317] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:37.781989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2305-f542-45f3-9639-cc68f986f570 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:37.782425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2305-f542-45f3-9639-cc68f986f570 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:37.782543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2305-f542-45f3-9639-cc68f986f570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:37.853181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a24d2305-f542-45f3-9639-cc68f986f570 None None] MFA Rules not processed for user `13fa28a8fed841f499f8ca013dd715e7`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:37.854574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a24d2305-f542-45f3-9639-cc68f986f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:37.923648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:37.985021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 668/1318] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:38.274673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:38.277333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:38.414891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bf68-3a1d-4dd5-92bf-6e412c09ba10 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:38.418195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:38.493315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:38.494047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:38.494388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:38.494791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:38.496424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:38.612728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:38.614772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57a350-b54b-4fc8-9afc-8694498c82d5 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:38.633110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 651/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:41.121569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.122591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.122591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.124699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 669/1320] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:41.131071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adecf7-6980-41df-9291-2b079688406a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.131821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adecf7-6980-41df-9291-2b079688406a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.131994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adecf7-6980-41df-9291-2b079688406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.178304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.213327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-76adecf7-6980-41df-9291-2b079688406a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:41.214574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-76adecf7-6980-41df-9291-2b079688406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:41.318454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 652/1321] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:41.623995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:41.625572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:41.743864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dadd34-7af4-436b-937c-6d5ec8e26fdf None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:41.830683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:41.831610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:41.832033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:41.832458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:41.835415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:41.849277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:41.964795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:41.966254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738bfc-9545-41de-bc9d-6952d89312db service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:41.986863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 670/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:44.540641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.541247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.541247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.542557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 653/1323] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:44.547893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51afe3-a0ee-4d75-8d80-949642fd61a7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:44.548347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51afe3-a0ee-4d75-8d80-949642fd61a7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:44.548465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51afe3-a0ee-4d75-8d80-949642fd61a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:44.617811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fb51afe3-a0ee-4d75-8d80-949642fd61a7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:44.619218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fb51afe3-a0ee-4d75-8d80-949642fd61a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:44.679432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:44.720757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 671/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:45.006850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:45.008590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:45.122584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d2d79-8e19-4700-8cc5-9fa7f21ed40b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:45.135538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:45.201859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:45.202475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:45.202754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:45.203038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:45.204661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:45.320925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:45.322636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9266f-71dc-4c14-80af-fecf7a99a624 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:45.342974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 654/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:47.329399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.329984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.329984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.331240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 672/1326] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:47.337261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefac6-ad04-4ac2-b71e-9a3fcb7431a8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:47.337936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefac6-ad04-4ac2-b71e-9a3fcb7431a8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:47.338405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eefac6-ad04-4ac2-b71e-9a3fcb7431a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:47.389314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:47.409555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f1eefac6-ad04-4ac2-b71e-9a3fcb7431a8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:47.410802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f1eefac6-ad04-4ac2-b71e-9a3fcb7431a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:47.521308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 655/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:47.804427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:47.806192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:47.925175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc219a7-60df-458f-b4f9-96342bbcf4e6 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:48.001159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:48.001773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:48.002055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:48.002293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:48.003971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:48.021987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:48.121994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:48.123503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2499e1f-aa27-4f18-a62f-33476b799c74 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:48.142740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 673/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:50.113571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.114187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.114187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.115613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 656/1329] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:50.121500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9c7c-7eef-4332-a57c-600963da7f4f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.122087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9c7c-7eef-4332-a57c-600963da7f4f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.122087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa9c7c-7eef-4332-a57c-600963da7f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.194034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a7fa9c7c-7eef-4332-a57c-600963da7f4f None None] MFA Rules not processed for user `13fa28a8fed841f499f8ca013dd715e7`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:50.195295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a7fa9c7c-7eef-4332-a57c-600963da7f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:50.263614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:50.297584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 674/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:50.577655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:50.579272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:50.702501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9495-1b52-4c41-be54-4798d3cecbe3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:50.723825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:50.780952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:50.781480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:50.781751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:50.782023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:50.783672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:50.903901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:50.905444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f1b76-154c-4d94-92ea-9f4037ca1900 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:50.926014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 657/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:53.281819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.282406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.282406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.283757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 675/1332] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:53.289960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed39f9-2b37-4636-a986-4d82a415a08e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.290826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed39f9-2b37-4636-a986-4d82a415a08e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.291048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed39f9-2b37-4636-a986-4d82a415a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.359320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:53.363654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-cfed39f9-2b37-4636-a986-4d82a415a08e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:53.366174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cfed39f9-2b37-4636-a986-4d82a415a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:53.481796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 658/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:53.764363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:53.765949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:53.880888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb4b31-9d81-4902-afd2-fbbdad26b8ee None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:53.955041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:53.955670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:53.955948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:53.956216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:53.957814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:53.985259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:54.076207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:54.077573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780bc5-0f4d-4018-b4e2-79e08ee55b7f service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:54.097293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 676/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:56.068918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.069802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.069802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.072260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 659/1335] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:56.077813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70eaa0-e125-4605-b487-a0f13e31e847 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.078290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70eaa0-e125-4605-b487-a0f13e31e847 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.078388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70eaa0-e125-4605-b487-a0f13e31e847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.151918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3b70eaa0-e125-4605-b487-a0f13e31e847 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:56.153127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3b70eaa0-e125-4605-b487-a0f13e31e847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:56.242852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.270279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 677/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:56.550725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:56.552330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:56.679032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c4650-8a30-46db-afe2-5e711648d496 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:56.717330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:56.766081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:56.766691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:56.766971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:56.767276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:56.768856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:56.880657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:56.882088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbac477-748c-43f5-9a32-0918a60169de service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:56.901019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 660/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:47:59.103825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.104714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.104714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.106870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 678/1338] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:47:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:47:59.112307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff49b-8bd4-4a92-9046-3f55a460967a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.112739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff49b-8bd4-4a92-9046-3f55a460967a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.112831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ff49b-8bd4-4a92-9046-3f55a460967a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.180558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-880ff49b-8bd4-4a92-9046-3f55a460967a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:59.181956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-880ff49b-8bd4-4a92-9046-3f55a460967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:59.190512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.304646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 661/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:47:59.602041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:59.605547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:59.728579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e50e3-6242-410b-acbb-421e91fd8822 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:59.809260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:59.809860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:59.810133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:59.810380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:59.812149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:59.854888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:47:59.937803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.939310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b221f6f-5825-4d85-86e7-cfe0ea3c8866 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.958713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 679/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:47:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:03.891684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.892244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.892244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.893622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 662/1341] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:03.899029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecfb2-efc0-49a4-9d7d-6982f2958b85 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.899542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecfb2-efc0-49a4-9d7d-6982f2958b85 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.899542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ecfb2-efc0-49a4-9d7d-6982f2958b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.968108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9c6ecfb2-efc0-49a4-9d7d-6982f2958b85 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:03.969231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9c6ecfb2-efc0-49a4-9d7d-6982f2958b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:04.059838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.073120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 680/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:04.350170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.351758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:04.464320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4443d-7d3e-4f82-8004-2793f6f204fc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.503925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.543110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.543714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.544016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.544270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.545849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:04.665173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.666602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bace1-81b4-4604-aaa9-c7d15f76d931 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.686363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 663/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:07.461825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f81d68-1753-471f-b212-0da342470b12 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.462411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f81d68-1753-471f-b212-0da342470b12 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.462539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f81d68-1753-471f-b212-0da342470b12 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.464006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 681/1344] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:07.469549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306ac-857c-4d86-b3c1-11dc5f72f05c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.470036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306ac-857c-4d86-b3c1-11dc5f72f05c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.470228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223306ac-857c-4d86-b3c1-11dc5f72f05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.539131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-223306ac-857c-4d86-b3c1-11dc5f72f05c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:07.540226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-223306ac-857c-4d86-b3c1-11dc5f72f05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:07.560352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.645343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 664/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:07.944309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f81d68-1753-471f-b212-0da342470b12 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.945882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-55f81d68-1753-471f-b212-0da342470b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:08.058421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f81d68-1753-471f-b212-0da342470b12 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.144477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.145142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.145490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.145823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.147590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:08.200634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:08.276421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.277962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753e81-2956-4b46-9920-9f0289338329 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.298682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 682/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:12.317504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.318025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.318025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.319770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 665/1347] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:12.325301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdee1-2d4e-4d9c-9099-8a308700054a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.325841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdee1-2d4e-4d9c-9099-8a308700054a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.325841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bdee1-2d4e-4d9c-9099-8a308700054a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.402659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-303bdee1-2d4e-4d9c-9099-8a308700054a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:12.403985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-303bdee1-2d4e-4d9c-9099-8a308700054a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:12.510402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.513317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 683/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:12.797885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.799688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:12.919612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffbf34-8a2d-435e-a5ef-2392ac6623f9 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.967857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.996670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.997204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.997475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.997743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.999446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:13.113351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:13.114806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae8f72-a48c-4ee7-8329-0f220da83c64 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:13.135570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 666/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:18.075675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27bb4e-0ce6-45a7-8203-ff3096a41504 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.076260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27bb4e-0ce6-45a7-8203-ff3096a41504 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.076260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27bb4e-0ce6-45a7-8203-ff3096a41504 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.077889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 684/1350] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.083682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae84272-bfb3-42ea-8927-76d68139a9f7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.084088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae84272-bfb3-42ea-8927-76d68139a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.084281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae84272-bfb3-42ea-8927-76d68139a9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.181897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5ae84272-bfb3-42ea-8927-76d68139a9f7 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.183161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5ae84272-bfb3-42ea-8927-76d68139a9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:18.218796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.329614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 667/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:22.529012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923285-2f5a-47ac-a885-d3f7157fe2be None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.529577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923285-2f5a-47ac-a885-d3f7157fe2be None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.529577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923285-2f5a-47ac-a885-d3f7157fe2be None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.531250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 685/1352] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:22.536461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273462c-406a-4bce-a5a7-730c87e1e280 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.536951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273462c-406a-4bce-a5a7-730c87e1e280 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.537233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273462c-406a-4bce-a5a7-730c87e1e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.627380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7273462c-406a-4bce-a5a7-730c87e1e280 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:22.628807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7273462c-406a-4bce-a5a7-730c87e1e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:22.708711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.748885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 668/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:24.035563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc5b7-e217-4d36-894c-cfa325ad8cff None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.036142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc5b7-e217-4d36-894c-cfa325ad8cff None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.036236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc5b7-e217-4d36-894c-cfa325ad8cff None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.037563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 686/1354] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:24.042850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d36a-0a5d-468d-81d7-8dc7a20d0046 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.043353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d36a-0a5d-468d-81d7-8dc7a20d0046 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.043477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d36a-0a5d-468d-81d7-8dc7a20d0046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.127902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-11f7d36a-0a5d-468d-81d7-8dc7a20d0046 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:24.129145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-11f7d36a-0a5d-468d-81d7-8dc7a20d0046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:24.252096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 669/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:48:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:27.593963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca522-33b7-4b83-9afc-ac4b90498d9d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.594542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca522-33b7-4b83-9afc-ac4b90498d9d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.594542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca522-33b7-4b83-9afc-ac4b90498d9d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.595930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 687/1356] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:27.601353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf493-6bc5-423e-aad2-935d19b5183d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.601771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf493-6bc5-423e-aad2-935d19b5183d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.601885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf493-6bc5-423e-aad2-935d19b5183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.643146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.680896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-bcabf493-6bc5-423e-aad2-935d19b5183d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:27.682078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bcabf493-6bc5-423e-aad2-935d19b5183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:27.829445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 670/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:28.141657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.141953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.142219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.143836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 688/1358] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 30 00:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:28.153344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa9ab-dd8e-4072-904b-75969e4b9258 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.154165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa9ab-dd8e-4072-904b-75969e4b9258 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.155165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa9ab-dd8e-4072-904b-75969e4b9258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.240808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f75aa9ab-dd8e-4072-904b-75969e4b9258 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:28.241995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f75aa9ab-dd8e-4072-904b-75969e4b9258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:28.305939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.389348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 671/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:48:28.397589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:28.400770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:28.567467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edf95b-43e7-45d3-8bff-25dde0e83047 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.645372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.688435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.689119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.689466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.689796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.691489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:28.785185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.786933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99a63-7ef5-4617-aefe-c080b4725125 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.806423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 689/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:30.923762 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.924313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.924656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.926329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 672/1361] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:30.932474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd08ae-03f8-46d3-9177-e375bd26936c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.933018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd08ae-03f8-46d3-9177-e375bd26936c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.933274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd08ae-03f8-46d3-9177-e375bd26936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.006730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f8bd08ae-03f8-46d3-9177-e375bd26936c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:31.008270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f8bd08ae-03f8-46d3-9177-e375bd26936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.047880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.115038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 690/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:48:31.417337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.418890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.594937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2ebe-2a14-42a8-ad85-3fabcecede43 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.605894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.718656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.719282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.719480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.719740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.721346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.822522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.824265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fec344-c21b-41cd-a1fe-0ab572ad1889 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.843569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 673/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:34.113621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89cc0a-add6-4cfb-8fde-975ace63b98e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.114211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89cc0a-add6-4cfb-8fde-975ace63b98e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.114211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89cc0a-add6-4cfb-8fde-975ace63b98e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.115552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 691/1364] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:34.121206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280fde-a038-49c8-89e7-eb8423250049 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.121694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280fde-a038-49c8-89e7-eb8423250049 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.121813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280fde-a038-49c8-89e7-eb8423250049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.138400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.199162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-19280fde-a038-49c8-89e7-eb8423250049 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:34.200418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-19280fde-a038-49c8-89e7-eb8423250049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.309229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 674/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:48:34.605549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.605803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.606239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.608026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 692/1366] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 30 00:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:34.613603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4eac-0b53-4323-b308-18a1a166281b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.614074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4eac-0b53-4323-b308-18a1a166281b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.614556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4eac-0b53-4323-b308-18a1a166281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.695576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-daac4eac-0b53-4323-b308-18a1a166281b None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:34.697115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-daac4eac-0b53-4323-b308-18a1a166281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.734535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.822830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 675/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 30 00:48:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:34.831117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.833032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.997363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfe68-2836-43c0-b262-3049aa01880e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.019914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.118954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.119652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.119922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.120108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.121703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:35.216729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.218412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db9a41-5aa5-42b2-b4f7-801d47a8959d service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.242361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 693/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:35.817636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a6a-43ce-499d-aedf-5e1574f8c7b2 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.818217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a6a-43ce-499d-aedf-5e1574f8c7b2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.818217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a6a-43ce-499d-aedf-5e1574f8c7b2 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.819754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 676/1369] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:35.825187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2f5e-9545-4cbb-a581-3a7fc03921c2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.825448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2f5e-9545-4cbb-a581-3a7fc03921c2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.825732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2f5e-9545-4cbb-a581-3a7fc03921c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.859947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.899772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-404e2f5e-9545-4cbb-a581-3a7fc03921c2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:35.901134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-404e2f5e-9545-4cbb-a581-3a7fc03921c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:36.004348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 694/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:36.590955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.590955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.590955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.590955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 677/1371] 10.222.0.17 () {54 vars in 964 bytes} [Tue Aug 30 00:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:36.596272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b333-bd20-489b-8382-28ee119b76d8 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.596786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b333-bd20-489b-8382-28ee119b76d8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.596871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b333-bd20-489b-8382-28ee119b76d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.690311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7b68b333-bd20-489b-8382-28ee119b76d8 None None] MFA Rules not processed for user `87af1d19163043ffaa3412487446b02f`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:36.691630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7b68b333-bd20-489b-8382-28ee119b76d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:36.740421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.772377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 695/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Aug 30 00:48:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:36.780798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.782353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:36.900673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccc45d-bebe-4a0b-83a6-97c771156fd3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.952637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.995685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.996328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.996680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.996964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.998609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.135846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:37.137416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258445-f6e6-44d2-96cd-63e32cae8793 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.166893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 678/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:37.255134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.255446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.255591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.257253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 696/1374] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:37.262181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9a2bf-794f-43be-835e-bdb3ce20a54d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.262502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9a2bf-794f-43be-835e-bdb3ce20a54d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.262789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9a2bf-794f-43be-835e-bdb3ce20a54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.362421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3af9a2bf-794f-43be-835e-bdb3ce20a54d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:37.364040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3af9a2bf-794f-43be-835e-bdb3ce20a54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.386977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.476463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 679/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:37.773483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.775879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.936742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a53a52-1c9a-4464-ba60-701f7cdebbab None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:37.989231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.053635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.054195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.054434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.054710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.056436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:38.146495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.148179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279997b-d908-4c21-9230-04c7b6c92881 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.167403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 697/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:40.128682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.129275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.129275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.130810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 680/1377] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:40.136545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c74c-8ca5-41b4-a6f9-3c94e4fb0aa5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.137011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c74c-8ca5-41b4-a6f9-3c94e4fb0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.137109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4c74c-8ca5-41b4-a6f9-3c94e4fb0aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.217526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-60c4c74c-8ca5-41b4-a6f9-3c94e4fb0aa5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:40.218740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-60c4c74c-8ca5-41b4-a6f9-3c94e4fb0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:40.225672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.330178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 698/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:40.620845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.622374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:40.818682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c077195-85d5-4ec6-bdc5-ff4885719e95 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.824177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.943486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.944103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.944442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.944819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.946609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:41.045992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.047748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b28a5-c389-46a9-a85d-cd66c4b1afa9 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.067861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 681/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:48:43.021324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.022283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.022788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.025357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 699/1380] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:43.031192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bb2b-68ac-496b-aeed-e3d4bfe48f49 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.031712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bb2b-68ac-496b-aeed-e3d4bfe48f49 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.031801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5bb2b-68ac-496b-aeed-e3d4bfe48f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.044905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.107544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d4f5bb2b-68ac-496b-aeed-e3d4bfe48f49 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:43.108863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d4f5bb2b-68ac-496b-aeed-e3d4bfe48f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:43.214587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 682/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:43.500658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.502269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:43.668504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.679970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd0bdd-3bb6-4644-8abc-f51b89dec691 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.801283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.801909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.802269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.802560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.804379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:43.903342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.905363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370751e7-8d83-4abc-86d8-d25dc69fe2e1 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.916738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.927713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 700/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:45.916672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.917257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.917257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.918658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 683/1383] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:45.924912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0057c-f072-4804-b915-fb0eaced63c6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.925467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0057c-f072-4804-b915-fb0eaced63c6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.926077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0057c-f072-4804-b915-fb0eaced63c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.007479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-86a0057c-f072-4804-b915-fb0eaced63c6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:46.008660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-86a0057c-f072-4804-b915-fb0eaced63c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:46.112494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 701/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:46.401257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.403919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:46.518938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.592521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a9803-7a9b-4c79-aebe-562d75672233 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.721305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.722757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.723068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.723354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.725032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:46.782524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.826068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.827792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c82f3b-d18b-4079-b282-9037fcfed9a5 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.847149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 684/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:48.832071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.832824 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.832824 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.834100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 702/1386] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:48.841492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5de3-9480-4ad4-98f2-a9ed1274a57e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.842112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5de3-9480-4ad4-98f2-a9ed1274a57e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.842277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5de3-9480-4ad4-98f2-a9ed1274a57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.923490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-54df5de3-9480-4ad4-98f2-a9ed1274a57e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:48.924819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-54df5de3-9480-4ad4-98f2-a9ed1274a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.057915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 685/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:49.354366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.356148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.453140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.541146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7fa03-c8f3-4838-b910-ca4016b07ca5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.676124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.676773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.677050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.677328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.678999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.721946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.786776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:49.789204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c424985-553f-4024-9e91-61c9f163d5db service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:49.813011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 703/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:51.767353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.767911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.767911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.769249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 686/1389] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:51.775055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f441c-e21e-4ae3-82a4-1b150cb4cc49 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.775491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f441c-e21e-4ae3-82a4-1b150cb4cc49 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.775598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f441c-e21e-4ae3-82a4-1b150cb4cc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.845589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8b7f441c-e21e-4ae3-82a4-1b150cb4cc49 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:51.846797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8b7f441c-e21e-4ae3-82a4-1b150cb4cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:51.950274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 704/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:52.242465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.244035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.276482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.364334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357cc4-b353-42a3-88a8-e26d826f130e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.364599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357cc4-b353-42a3-88a8-e26d826f130e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.364896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357cc4-b353-42a3-88a8-e26d826f130e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.366272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 705/1391] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:52.371389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1ee2-a559-4e0b-8041-ecd1241287b9 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.371671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1ee2-a559-4e0b-8041-ecd1241287b9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.371942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f1ee2-a559-4e0b-8041-ecd1241287b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.388435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.402276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7a20c-e738-431a-a090-43290ef267fd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.459109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-170f1ee2-a559-4e0b-8041-ecd1241287b9 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.460333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-170f1ee2-a559-4e0b-8041-ecd1241287b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.525378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.537832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.538454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.538729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.539046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.541450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.567042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 706/1392] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:48:52.639772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.641427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51084a73-ce6f-4e70-a619-7ce3de65c1f7 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.663618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 687/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:52.887630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.888216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.888216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.889729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 707/1394] 10.222.0.17 () {54 vars in 967 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:52.895358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ba060-1949-4612-9f7b-cba52ae50937 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.895836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ba060-1949-4612-9f7b-cba52ae50937 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.895945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ba060-1949-4612-9f7b-cba52ae50937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.106267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2d3ba060-1949-4612-9f7b-cba52ae50937 None None] MFA Rules not processed for user `97d15d2823be4939a617603cfdad0aef`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:53.107438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2d3ba060-1949-4612-9f7b-cba52ae50937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:53.170675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.201649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 688/1395] 10.222.0.17 () {58 vars in 1064 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:53.210020 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.211623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:53.315975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.323746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d902-a0eb-4739-94ab-2b248be5bcde None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.401788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.402480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.402758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.403231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.404897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:53.528068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.529532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982cc25-ab7b-4868-bf05-4fd48fa19f39 service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.549029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 708/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:54.301088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3a0b9-525a-4c04-a918-8158859a6713 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.301596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3a0b9-525a-4c04-a918-8158859a6713 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.301717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3a0b9-525a-4c04-a918-8158859a6713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.369605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-72e3a0b9-525a-4c04-a918-8158859a6713 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:54.370893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-72e3a0b9-525a-4c04-a918-8158859a6713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:54.460245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 689/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:48:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:48:54.507234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc50d0-25e6-4faf-a506-590dfaa18909 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.507832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc50d0-25e6-4faf-a506-590dfaa18909 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.508021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc50d0-25e6-4faf-a506-590dfaa18909 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.509495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 709/1398] 10.222.0.17 () {54 vars in 943 bytes} [Tue Aug 30 00:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:54.515843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.516215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.516550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.595619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] MFA Rules not processed for user `b98ce88b48c34731a1eae85d23de837a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:54.596827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:54.610195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:54.693979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f7920-5b8a-4515-bd54-4226d1243d99 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.694289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f7920-5b8a-4515-bd54-4226d1243d99 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.694564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f7920-5b8a-4515-bd54-4226d1243d99 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.696082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 710/1399] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:54.701383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.701670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.701942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.705993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 690/1400] 10.222.0.17 () {58 vars in 1040 bytes} [Tue Aug 30 00:48:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:54.716029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:54.726068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3293f1f-3b0a-4eec-921b-d357faf2d869 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:54.731697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:54.772053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:54.773673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:54.831218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:54.832190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.832462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.832727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.834342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:54.852362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:54.878342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 711/1401] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:54.923105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:54.924719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775cad9-aaca-4a29-8b61-55c212476d44 service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:54.933178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 691/1402] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:48:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:48:55.177082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:55.178297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:55.293939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.312051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc131-57df-4a29-ac08-683ddf339cd1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:55.442827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:55.443642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.444008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.444368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.446319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:55.547510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:55.549435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928d462-fd8b-4828-8d8d-aeefdbf7fce6 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:55.563457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.573884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 712/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:48:57.662541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.663172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.663172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.664787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 692/1404] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:57.670540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094e85-1fb9-4323-9330-18afb4e3153b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.671061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094e85-1fb9-4323-9330-18afb4e3153b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.671061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39094e85-1fb9-4323-9330-18afb4e3153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.747730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-39094e85-1fb9-4323-9330-18afb4e3153b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:57.749293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-39094e85-1fb9-4323-9330-18afb4e3153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:57.870435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 713/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:48:58.161328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.162878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:58.335886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2e9a6-bbfa-4a5c-b6d1-e6c38bf9cd7d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.351019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.481543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.482191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.482823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.483130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.484711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:58.584720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.586427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc58b5-1318-4bfa-bc3b-7c47337a9f57 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.610461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 693/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:00.599200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.599750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.599750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.601393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 714/1407] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:00.612453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2bb6-55ac-4dc1-bf84-e00dafc263b5 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.613027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2bb6-55ac-4dc1-bf84-e00dafc263b5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.613027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f2bb6-55ac-4dc1-bf84-e00dafc263b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.634036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.689377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-234f2bb6-55ac-4dc1-bf84-e00dafc263b5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:00.690665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-234f2bb6-55ac-4dc1-bf84-e00dafc263b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:00.794070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 694/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:01.077428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.078978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:01.168264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.247574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38030b-cb06-4fd2-a808-b9cd41fd354a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.373794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.374411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.374687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.374952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.376731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:01.421187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.485357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.487141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83316fec-aab4-413f-9538-bbde03cbd5e6 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.508519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 715/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:10.792076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=58 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:11.505862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=29 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.413446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489896f5-c28b-44b5-a80b-7fef9c9d07b3 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.414055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489896f5-c28b-44b5-a80b-7fef9c9d07b3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.414055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489896f5-c28b-44b5-a80b-7fef9c9d07b3 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.415654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 695/1410] 10.222.0.17 () {54 vars in 918 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.420917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee08404-896b-4974-b67f-8d02b98fba77 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.421381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee08404-896b-4974-b67f-8d02b98fba77 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.421476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee08404-896b-4974-b67f-8d02b98fba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.506635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dee08404-896b-4974-b67f-8d02b98fba77 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:17.507906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dee08404-896b-4974-b67f-8d02b98fba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:17.636938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 716/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:17.651982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7d48-8a33-41fe-9429-5527a4de0db8 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.652265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7d48-8a33-41fe-9429-5527a4de0db8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.652533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7d48-8a33-41fe-9429-5527a4de0db8 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.653933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 696/1412] 10.222.0.17 () {54 vars in 918 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.658731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30dda-c009-4219-a992-86bc5659c2a4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.659044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30dda-c009-4219-a992-86bc5659c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.659308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30dda-c009-4219-a992-86bc5659c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.736169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.739945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4fb30dda-c009-4219-a992-86bc5659c2a4 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:17.742036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4fb30dda-c009-4219-a992-86bc5659c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:17.875917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 717/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:20.275566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.276207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.276207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.277632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:20.283193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622036e-0b71-4e23-8b28-6eff5f2b0a4c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.283644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622036e-0b71-4e23-8b28-6eff5f2b0a4c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.283738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622036e-0b71-4e23-8b28-6eff5f2b0a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.355656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4622036e-0b71-4e23-8b28-6eff5f2b0a4c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:20.356968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4622036e-0b71-4e23-8b28-6eff5f2b0a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:20.418532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.470005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:20.760130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.762376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:20.918608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c372-a55a-4f2c-95a8-2a1eb35e0818 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.995909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.038630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.039275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.039556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.039833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.041491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:21.150610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.152252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c446-efe2-49f0-9968-299afbcfdf07 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.172191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 698/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:22.066100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.067088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.067088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.069158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:22.076197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674aadd-5cde-4a70-8c0b-b5e88147d8ef None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.076615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674aadd-5cde-4a70-8c0b-b5e88147d8ef None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.076734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674aadd-5cde-4a70-8c0b-b5e88147d8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.160391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e674aadd-5cde-4a70-8c0b-b5e88147d8ef None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:22.161597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e674aadd-5cde-4a70-8c0b-b5e88147d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:22.190594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.262862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:22.411468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.412982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:22.577515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54a7ab-ee49-462a-b8b5-5f1563a93ca0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.624609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.702631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.703269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.703564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.703815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.705430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:22.815625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.817263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e07693-b0a5-4a7c-8da5-c6a8b31ce211 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.838052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 720/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:23.788763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.789605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.789605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.791819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:23.799589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04116-bf74-46c9-a27c-e9fb93884ad6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.800037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04116-bf74-46c9-a27c-e9fb93884ad6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.800150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04116-bf74-46c9-a27c-e9fb93884ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.857951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.886090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8ba04116-bf74-46c9-a27c-e9fb93884ad6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:23.887497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8ba04116-bf74-46c9-a27c-e9fb93884ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:23.994548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:49:24.155083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.157545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:24.325786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfbe62-ff36-4934-8cbd-869749a4ab62 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.339905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.453834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.454459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.454738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.455036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.457062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:24.563713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.565312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f33d9-8cc6-4b15-8df2-23e9a87d3fdc service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.586556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 701/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:25.470290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.470913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.470913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.472290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:25.478473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c21f9d-0916-405a-944f-7c8eba47134c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.478997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c21f9d-0916-405a-944f-7c8eba47134c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.479372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c21f9d-0916-405a-944f-7c8eba47134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.504868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.567184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-26c21f9d-0916-405a-944f-7c8eba47134c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:25.568436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-26c21f9d-0916-405a-944f-7c8eba47134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:25.681869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:25.838426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.840043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:25.975763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.016900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46d1f1-f91f-42a9-bc3e-14508beaad4a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.144972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.145587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.145872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.146138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.147801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:26.230454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.245934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.247598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b323f89-cacd-41d6-9ce6-6daf282d802f service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.267972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 723/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:27.142126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.142709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.142709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.144096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:27.150424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e757b-87df-47e4-9169-57d3c49656de None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.151366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e757b-87df-47e4-9169-57d3c49656de None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.151366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e757b-87df-47e4-9169-57d3c49656de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.246764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-320e757b-87df-47e4-9169-57d3c49656de None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:27.248015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-320e757b-87df-47e4-9169-57d3c49656de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:27.350284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:27.505232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.507806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:27.612149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.682881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec417c4-839d-44a0-a84c-c1465d3bdaf6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.803820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.804460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.804734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.804997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.806605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:27.866473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.917000 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.918604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d79c7-0fb6-4fee-b2ee-15a0b60786a5 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.939158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 704/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:28.747536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.748409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.748409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.750685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:28.756721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3409c-8d37-4d9f-9b4d-771aa75f3172 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.757437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3409c-8d37-4d9f-9b4d-771aa75f3172 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.758359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3409c-8d37-4d9f-9b4d-771aa75f3172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.844176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-06d3409c-8d37-4d9f-9b4d-771aa75f3172 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:28.845379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-06d3409c-8d37-4d9f-9b4d-771aa75f3172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:28.970532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:29.129820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.132359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:29.199035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.325806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1da07c-98d7-4d32-ab12-585f0da1ec50 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.443423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.444048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.444327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.444596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.446194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:29.453508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.550379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.552067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d895cf9-6cf2-4b4e-837a-af7f7862d8fd service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.570758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 726/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:30.390793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.391443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.391443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.392838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:30.398548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23f8bf-f2ba-42bb-a041-d964deda7bdc None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.399089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23f8bf-f2ba-42bb-a041-d964deda7bdc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.399089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23f8bf-f2ba-42bb-a041-d964deda7bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.483061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5d23f8bf-f2ba-42bb-a041-d964deda7bdc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:30.484291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5d23f8bf-f2ba-42bb-a041-d964deda7bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:30.557124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.589267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:30.751045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.753516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:30.774776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.912564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7acba0-63a8-4a78-b24d-81fe30cae660 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.012381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.048314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.048922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.049219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.049480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.051061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:31.157678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.159414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f5594-cf4f-4a53-8e19-255e65d7493b service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.179071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 707/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:31.991092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.991715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.991715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.993006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:32.001915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124fee-acb8-440a-ad34-ad874429615d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.002438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124fee-acb8-440a-ad34-ad874429615d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.002538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124fee-acb8-440a-ad34-ad874429615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.100303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f5124fee-acb8-440a-ad34-ad874429615d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:32.101524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f5124fee-acb8-440a-ad34-ad874429615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:32.147102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.207872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:32.362313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.363886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:32.536373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ae2d-f1d2-487a-b92b-dec5a31f1a91 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.595827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.656068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.656591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.656862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.657126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.658729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:32.755164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.756819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2e05-1abe-4a24-8b17-0968e93c9d5f service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.776870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 729/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:33.603664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.604583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.604583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.606834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:33.615238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505caa98-3277-4b68-b637-0fd51c8e6110 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.615754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505caa98-3277-4b68-b637-0fd51c8e6110 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.615849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505caa98-3277-4b68-b637-0fd51c8e6110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.702507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-505caa98-3277-4b68-b637-0fd51c8e6110 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:33.703677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-505caa98-3277-4b68-b637-0fd51c8e6110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:33.706129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.804236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:33.959870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.961314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:34.127437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1625f71-0fb8-4a09-94e8-20b433792a0f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.156825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.248661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.249185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.249459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.249731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.251359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:34.345229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.346815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecca81-ffac-4272-af5b-07a7cf95bc20 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.365253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 710/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:36.201708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.202327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.202327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.203904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:36.210907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838915-d5fc-4fa9-b5af-fcd68835eb89 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.211662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838915-d5fc-4fa9-b5af-fcd68835eb89 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.211840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838915-d5fc-4fa9-b5af-fcd68835eb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.262702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.293547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-40838915-d5fc-4fa9-b5af-fcd68835eb89 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:36.294795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-40838915-d5fc-4fa9-b5af-fcd68835eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:36.423228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:36.723160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.725607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:36.841650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69006a5c-a3c6-4701-bdbb-47327dca4d8d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.884090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.930382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.930994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.931311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.931576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.933705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:37.061338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.062716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc098116-dfbe-4135-b4ff-35c3ca83f543 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.081910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 732/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:39.161754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.162355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.162355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.163730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:39.170245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1c04-9e2c-4a16-8227-627c44878c4b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.171048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1c04-9e2c-4a16-8227-627c44878c4b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.171219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1c04-9e2c-4a16-8227-627c44878c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.252379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-88eb1c04-9e2c-4a16-8227-627c44878c4b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:39.253595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-88eb1c04-9e2c-4a16-8227-627c44878c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:39.269155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.362795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:39.653240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.655663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:39.769439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce4ba5-5af5-47ca-817c-ed76775333d4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.782134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.854361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.855024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.855325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.855604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.857256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:39.975559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.977010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f4c70-b39c-4e50-91b4-83d83b2c8137 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.997043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 713/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:42.332417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.333047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.333047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.334339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:42.340305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe420d-761c-45b2-997a-56eb2d84c43b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.340768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe420d-761c-45b2-997a-56eb2d84c43b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.340863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe420d-761c-45b2-997a-56eb2d84c43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.391622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.416091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9fbe420d-761c-45b2-997a-56eb2d84c43b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:42.417459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9fbe420d-761c-45b2-997a-56eb2d84c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:42.518973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:42.824808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.826410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:42.944994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6ed3a-3bf9-4bad-aad0-1be9323506b8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.989095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.025457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.026096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.026354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.026643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.028274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:43.141423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.142873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d2f6c-35da-42d3-b110-3dafc467fe3d service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.162541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 735/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:45.252179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.252759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.252759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.254943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:45.263921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac12c-d797-4c55-b0d2-9a4abf7e09bc None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.264653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac12c-d797-4c55-b0d2-9a4abf7e09bc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.264826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac12c-d797-4c55-b0d2-9a4abf7e09bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.351548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fd9ac12c-d797-4c55-b0d2-9a4abf7e09bc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:45.352837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fd9ac12c-d797-4c55-b0d2-9a4abf7e09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:45.380729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.464047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:45.763290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.765781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:45.899315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab218911-98b6-44af-8de3-b4a762de2b00 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.921848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.979666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.980290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.980567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.980835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.982451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:46.110811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:46.112319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d87ade-c85f-4692-8e15-0ae00ac5941a service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:46.132388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 716/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:48.095413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.096317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.096664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.099184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:48.106697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49596354-2f42-4e0f-afc3-9fb7b67072ff None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.107198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49596354-2f42-4e0f-afc3-9fb7b67072ff None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.107299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49596354-2f42-4e0f-afc3-9fb7b67072ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.200772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.204505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-49596354-2f42-4e0f-afc3-9fb7b67072ff None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:48.206737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-49596354-2f42-4e0f-afc3-9fb7b67072ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.320189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:48.611726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.614218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.736566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659fc75-e455-4da6-9c8f-d6bbe0cee95d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.789422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.815602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.816212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.816484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.816743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.818380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.931186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.932867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275c19-dd1a-43c4-8dc9-b9a71fb7d0f8 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.955573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 738/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:50.916087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.916914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.917004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.919208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:50.924602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfcef-4beb-4afd-ad24-1e843386a8c7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.925047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfcef-4beb-4afd-ad24-1e843386a8c7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.925152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfcef-4beb-4afd-ad24-1e843386a8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.006208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f09dfcef-4beb-4afd-ad24-1e843386a8c7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:51.007556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f09dfcef-4beb-4afd-ad24-1e843386a8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:51.041660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.120701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:51.405023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.406657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:51.517104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686c23-d1e9-4670-b1ff-dfe9b857d941 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.549296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.599028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.599825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.600099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.600369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.601957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:51.721619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.723127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf5875-e385-4b90-8bb8-30c5d0efbd79 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.743674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 719/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:53.807753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.808301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.808301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.809585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:53.816520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eade-d767-488f-a9f7-9c7afd7740a1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.817012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eade-d767-488f-a9f7-9c7afd7740a1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.817105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eade-d767-488f-a9f7-9c7afd7740a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.899522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d273eade-d767-488f-a9f7-9c7afd7740a1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:53.900750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d273eade-d767-488f-a9f7-9c7afd7740a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:53.909475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.005894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:54.302882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.305497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:54.424534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696a453-423d-451a-ae67-9fd59fdd42ab None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.492492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.512761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.513377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.513649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.513914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.515584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:54.635375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.637392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d961d59-f364-4beb-867c-b33e951f8832 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.657175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 741/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:56.626463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.627061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.627061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.628651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:56.634829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bf0-8edf-4408-bdd5-08f222bc8aae None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.635413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bf0-8edf-4408-bdd5-08f222bc8aae None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.635413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bf0-8edf-4408-bdd5-08f222bc8aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.715186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-db0a7bf0-8edf-4408-bdd5-08f222bc8aae None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:56.716424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-db0a7bf0-8edf-4408-bdd5-08f222bc8aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:56.773085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.853123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:49:57.155971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.158451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:57.276432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22816d90-29ed-4bd2-be53-7f4472915e5f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.316825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.362789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.363429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.363703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.363970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.365556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:57.494798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.496493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6c5f0-6bba-45b6-8909-cb0d3e771379 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.515743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 722/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:49:59.477465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.478430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.478430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.480392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:49:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:59.486481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d605a-c1c5-44b2-a841-93d4b0d7615e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.486937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d605a-c1c5-44b2-a841-93d4b0d7615e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.487093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d605a-c1c5-44b2-a841-93d4b0d7615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.574616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f16d605a-c1c5-44b2-a841-93d4b0d7615e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:59.576257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f16d605a-c1c5-44b2-a841-93d4b0d7615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:59.599605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:59.703667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:49:59.984663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:59.986273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:00.101880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898201a-203c-4b24-9ce4-e2f99b5ded05 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.175297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.186649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.186649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.186649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.186649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.187961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:00.305345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.307230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e671d-cc58-4071-a6e2-b7446ba5e9d1 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.327948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 744/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:02.270277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.271274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.271274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.273409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:50:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:02.280140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3761e-ae72-4b5e-b188-c0992996a718 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.280929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3761e-ae72-4b5e-b188-c0992996a718 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.281086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3761e-ae72-4b5e-b188-c0992996a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.352386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dfd3761e-ae72-4b5e-b188-c0992996a718 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:02.353596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dfd3761e-ae72-4b5e-b188-c0992996a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:02.406614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.466544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:02.748747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.750338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:02.863163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366640a1-5d32-4a0e-b310-f5caf676257c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.916309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.942598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.943239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.943517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.943784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.945375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:03.067776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.069235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a896bf8-eb31-48eb-b847-cea53c24d1b9 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.098676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 725/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:12.462601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=22 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.094404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=66 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:22.846574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0337703-25de-4b89-bbe2-2fc451e32a83 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.847587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0337703-25de-4b89-bbe2-2fc451e32a83 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.847587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0337703-25de-4b89-bbe2-2fc451e32a83 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.849843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 746/1471] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:22.859049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad4be-d29a-49bf-b2c0-285fa8276c03 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.859487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad4be-d29a-49bf-b2c0-285fa8276c03 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.859607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad4be-d29a-49bf-b2c0-285fa8276c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.937205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e53ad4be-d29a-49bf-b2c0-285fa8276c03 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:22.938416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e53ad4be-d29a-49bf-b2c0-285fa8276c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:23.060715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 726/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:23.362535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.362915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.363284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.365434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 747/1473] 10.222.0.17 () {54 vars in 967 bytes} [Tue Aug 30 00:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:23.372144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6443fb-17e1-4c4d-9439-6748f75e19ff None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.372455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6443fb-17e1-4c4d-9439-6748f75e19ff None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.372725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6443fb-17e1-4c4d-9439-6748f75e19ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.456058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0e6443fb-17e1-4c4d-9439-6748f75e19ff None None] MFA Rules not processed for user `97d15d2823be4939a617603cfdad0aef`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:23.457333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0e6443fb-17e1-4c4d-9439-6748f75e19ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:23.479371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.539196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 727/1474] 10.222.0.17 () {58 vars in 1064 bytes} [Tue Aug 30 00:50:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:23.547546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.549843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:23.665536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349bbd3-8c14-4bc2-ae43-bdd2f38667b6 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.759616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.760287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.760564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.760833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.762808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:23.826677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.887106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.889267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fef8b2-dc69-415e-94f7-30b01e8b4635 service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.909593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 748/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:50:25.920059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.920955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.920955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.923111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 728/1476] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:25.930038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc7ba2-be56-42fc-938e-cab452693c56 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.931201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc7ba2-be56-42fc-938e-cab452693c56 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.931695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc7ba2-be56-42fc-938e-cab452693c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.007044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9efc7ba2-be56-42fc-938e-cab452693c56 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:26.008276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9efc7ba2-be56-42fc-938e-cab452693c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:26.124699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 749/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:26.399371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.401638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:26.541938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50c5c6-fc20-4b0c-a2ea-04ea0227f147 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.596196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.646937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.647585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.647861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.648133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.648774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.650584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:50:26.650584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:50:26.650584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:50:26.651906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685bf00-f367-4738-b0b7-37fe281b31c5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.671227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 729/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/projects => generated 2553 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:50:28.616343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.616917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.616917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.618254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 750/1479] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:28.624734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fa84-2457-4db9-862d-c7fb4d98a4ae None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.625401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fa84-2457-4db9-862d-c7fb4d98a4ae None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.625501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fa84-2457-4db9-862d-c7fb4d98a4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.709725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-bf12fa84-2457-4db9-862d-c7fb4d98a4ae None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:28.710848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bf12fa84-2457-4db9-862d-c7fb4d98a4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:28.835289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 730/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:29.130181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.131889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:29.142771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.303977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77b79-144f-499b-98a7-22ceeccba827 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.396536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.431164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.431780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.432060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.432302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.434297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:29.538018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.539954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784c5f8-2875-4518-888c-54257580f735 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.560131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 751/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:31.702161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.703087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.703087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.705543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 731/1482] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:31.712159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b9e40-2292-4cc6-b04b-00deeeb6d72d None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.712913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b9e40-2292-4cc6-b04b-00deeeb6d72d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.713070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b9e40-2292-4cc6-b04b-00deeeb6d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.788317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dd7b9e40-2292-4cc6-b04b-00deeeb6d72d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:31.789530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dd7b9e40-2292-4cc6-b04b-00deeeb6d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:31.841610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:31.893086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 752/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:32.191834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.193482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:32.227123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.314024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067730-d088-4aa0-8c4d-8b59f3e03371 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.397388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.398007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.398284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.398578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.400249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:32.485475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.525141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.526855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786fd3b-ce0b-4837-9e66-962d20f21fe4 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.546791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 732/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:34.504329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.504936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.504936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.506285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 753/1485] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:34.512356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eac17-4883-44f7-ac2a-769749a4316f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.512812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eac17-4883-44f7-ac2a-769749a4316f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.512925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eac17-4883-44f7-ac2a-769749a4316f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.595980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e89eac17-4883-44f7-ac2a-769749a4316f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:34.597191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e89eac17-4883-44f7-ac2a-769749a4316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:34.702778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 733/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:34.993711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:34.996242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:35.108132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aedbc8-67b9-4a65-be75-cc68cf625c66 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:35.188089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:35.190787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:35.191826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:35.192873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:35.193443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:35.196532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:35.322210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:35.323666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4161d-bcc2-488f-b722-645e4fb3d7c9 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:35.344324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 754/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:37.451876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:37.452437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:37.452437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:37.454248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 734/1488] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:37.460041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265473-d276-4b02-96b7-833e91ec7c78 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:37.460503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265473-d276-4b02-96b7-833e91ec7c78 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:37.460612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f265473-d276-4b02-96b7-833e91ec7c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:37.535501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3f265473-d276-4b02-96b7-833e91ec7c78 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:37.536765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3f265473-d276-4b02-96b7-833e91ec7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:37.597901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:37.646668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 755/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:50:37.956287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:37.958123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:38.006107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:38.089690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992bd71-e069-4a76-bf65-a962870b04c5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:38.172562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:38.173180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:38.173460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:38.173727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:38.175398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:38.282300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:38.313859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:38.315337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a19ae7-52ef-4e66-84c3-8ee75112c025 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:38.336356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 735/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:40.523516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:40.524098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:40.524098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:40.525471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 756/1491] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 30 00:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:40.531402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55025f9f-47a7-4bd0-9596-1669ce11026c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:40.531899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55025f9f-47a7-4bd0-9596-1669ce11026c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:40.531899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55025f9f-47a7-4bd0-9596-1669ce11026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:40.601497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-55025f9f-47a7-4bd0-9596-1669ce11026c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:40.602716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-55025f9f-47a7-4bd0-9596-1669ce11026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:40.719605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 736/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:50:41.015306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:41.017872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:41.155763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87377252-ba6e-4905-82e4-8fb09b74eefc None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:41.238010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:41.238616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:41.238888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:41.239180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:41.240775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:41.250088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:41.375419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:41.376877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9969aa-efc5-4d3a-a0d7-f0b5a2841ca3 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:41.395683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 757/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:50:50.716077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=82 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:51.393225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=50 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.080039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.080956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.080956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.167447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:05.168661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:05.274260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 737/1494] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:52:05.283682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.284233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.284233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.369275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:05.370491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:05.471681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 758/1495] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:52:05.476604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.477827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:05.489835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6d961-9930-4505-8deb-058a81e55232 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.568193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.607656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.608277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.608558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.608827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.609480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:05.609777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.610856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:05.610856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:52:05.610856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:52:05.611779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.623634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 738/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:05.632728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.648046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ec0d9-bf0c-45d6-bd13-e3a20ea9ecc2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.730393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:05.765326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.765934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.766209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.766472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.767190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.769005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:05.769005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:52:05.769005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:52:05.770368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:05.829491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 759/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:52:05.834428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:05.836022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:05.878807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf7fac-4687-4c0b-8abf-80f4d0d795e8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:05.984751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:05.985361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:05.985632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:05.985899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:05.986523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:05.987565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:05.987565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:52:05.987565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:52:05.989070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:06.036958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 739/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:52:05 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:52:06.041358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:06.042943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:06.114597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb70a5-cd67-4d14-831c-d65144371b8e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:06.210797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:06.230540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:06.231754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.232214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.232660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.233753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:06.235526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:06.235526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:52:06.235526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:52:06.237194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:06.251335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 760/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:52:06.258330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:06.260895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:06.276195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c14d-9eb7-47f8-9529-ded6eb38092a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:06.304129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:06.380952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:06.381571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.381848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.382110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.382752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:06.383865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:06.383865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:52:06.383865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:52:06.384769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c5390-06f5-43ba-b056-4543040be2aa admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:06.399147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 740/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:52:06.403784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:06.405355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:06.445356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa01c6d-39c0-490b-89e7-13a4e9df27ee admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:06.547794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:06.548371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.548650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.548922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] PATH_INFO: `/v3/projects/db33fe7368464fe2ae67d32fe8b3a9bf/users/bc41aaf894ca45eca4584257fe1b1475/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.573427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db33fe7368464fe2ae67d32fe8b3a9bf, user_id=bc41aaf894ca45eca4584257fe1b1475, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:06.574513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:06.574513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:52:06.574513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:52:06.576875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bb647-9294-4a0a-826b-82c8fdde450b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:06.610562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 761/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:52:06 2022] PUT /identity/v3/projects/db33fe7368464fe2ae67d32fe8b3a9bf/users/bc41aaf894ca45eca4584257fe1b1475/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:52:06.623599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b2c5c-c898-46c3-95b2-c3c3cfde7a2a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.623881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b2c5c-c898-46c3-95b2-c3c3cfde7a2a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.624153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b2c5c-c898-46c3-95b2-c3c3cfde7a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.695803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6c2b2c5c-c898-46c3-95b2-c3c3cfde7a2a None None] MFA Rules not processed for user `bc41aaf894ca45eca4584257fe1b1475`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:06.697048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6c2b2c5c-c898-46c3-95b2-c3c3cfde7a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:06.780311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 741/1502] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:52:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:52:06.798225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.798399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.798698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.800107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 762/1503] 10.222.0.17 () {54 vars in 964 bytes} [Tue Aug 30 00:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:06.806058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c427d2-c390-42f2-849f-cd7642a2d2de None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:06.806394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c427d2-c390-42f2-849f-cd7642a2d2de None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:06.806770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c427d2-c390-42f2-849f-cd7642a2d2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:06.849760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:06.904717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-44c427d2-c390-42f2-849f-cd7642a2d2de None None] MFA Rules not processed for user `87af1d19163043ffaa3412487446b02f`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:06.906087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-44c427d2-c390-42f2-849f-cd7642a2d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:06.985952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 742/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Aug 30 00:52:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:52:06.997476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:06.999042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:07.053860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:07.124333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf540-b3fe-49b8-afcc-9494add8c578 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:07.218358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:07.218988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.219306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.219654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.221406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:07.335444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:07.336789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:07.340313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:07.355937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 763/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:52:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:52:07.409971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc98af-cce9-4751-9e64-3f556f3e1d91 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.410483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc98af-cce9-4751-9e64-3f556f3e1d91 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.410889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc98af-cce9-4751-9e64-3f556f3e1d91 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.412526 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 743/1506] 10.222.0.17 () {52 vars in 830 bytes} [Tue Aug 30 00:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:07.417553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:07.419344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:07.537001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d7b5-dcbd-4ee9-b0df-13e936b9de23 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:07.657128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:07.658004 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.658382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.658597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] PATH_INFO: `/v3/users/bc41aaf894ca45eca4584257fe1b1475` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.677461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc41aaf894ca45eca4584257fe1b1475)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:07.678511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:07.678511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:52:07.678511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:52:07.679797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:07.690477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:07.761530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc41aaf894ca45eca4584257fe1b1475'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:07.780620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc41aaf894ca45eca4584257fe1b1475'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:07.781218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc41aaf894ca45eca4584257fe1b1475'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:07.812452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc41aaf894ca45eca4584257fe1b1475'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:07.826093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc41aaf894ca45eca4584257fe1b1475'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:07.832489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 764/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:52:07 2022] DELETE /identity/v3/users/bc41aaf894ca45eca4584257fe1b1475 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:52:07.837135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a252-7737-4777-bdff-da188583e204 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:07.837422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a252-7737-4777-bdff-da188583e204 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:07.837690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a252-7737-4777-bdff-da188583e204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:07.902157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1e58a252-7737-4777-bdff-da188583e204 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:07.903676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1e58a252-7737-4777-bdff-da188583e204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:07.981941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:08.004268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 744/1508] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:52:08.014266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:08.015555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:08.131828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906154b7-ae67-4877-8f1a-cbd9bc9b510d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:08.328904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:08.356188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:08.357083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:08.357451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:08.357867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:08.360560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:08.489859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:08.491335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73153a1-4f4a-4b06-93f2-e2942c32b0a1 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:08.510354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 765/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:52:08.884030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a252-7737-4777-bdff-da188583e204 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:08.885295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1e58a252-7737-4777-bdff-da188583e204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:08.897615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a252-7737-4777-bdff-da188583e204 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:09.008151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:09.009047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:09.009421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:09.009821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] PATH_INFO: `/v3/projects/db33fe7368464fe2ae67d32fe8b3a9bf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:09.024169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db33fe7368464fe2ae67d32fe8b3a9bf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:09.025213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:52:09.025213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:52:09.025213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:52:09.026427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:09.091157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db33fe7368464fe2ae67d32fe8b3a9bf', 'request_id': 'req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:09.097677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db33fe7368464fe2ae67d32fe8b3a9bf', 'request_id': 'req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:09.109392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db33fe7368464fe2ae67d32fe8b3a9bf', 'request_id': 'req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:09.141630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] The token cache is being invalidate because project db33fe7368464fe2ae67d32fe8b3a9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:52:09.141821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:09.142248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-27ec1a27-a71d-40bf-acf5-60d92f05d6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:52:09.143713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 745/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:52:08 2022] DELETE /identity/v3/projects/db33fe7368464fe2ae67d32fe8b3a9bf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:52:18.507687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=62 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:19.139554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:38.807888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:38.808545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:38.808545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:38.809866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:38.815873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6654f5-b4fb-4a88-96f6-d5f591eafaac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:38.816356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6654f5-b4fb-4a88-96f6-d5f591eafaac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:38.816464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6654f5-b4fb-4a88-96f6-d5f591eafaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:38.926968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e6654f5-b4fb-4a88-96f6-d5f591eafaac None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:38.928261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e6654f5-b4fb-4a88-96f6-d5f591eafaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:39.058327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:39.216580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:39.219044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:39.357824 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fcd15-b9a3-47dc-bf55-80b184d07ceb None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:39.367988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:39.368737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:39.369116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:39.369559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:39.378971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:39.380598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191f04c-98de-42e2-b43a-01acf271c757 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:39.390446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 767/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:40.219598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8038-b744-4f01-94ec-3d150948e0be None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.220185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8038-b744-4f01-94ec-3d150948e0be None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.220185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8038-b744-4f01-94ec-3d150948e0be None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.221624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:40.227805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513f4e-0b42-4e63-a951-147f1781ef97 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.228562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513f4e-0b42-4e63-a951-147f1781ef97 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.228709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513f4e-0b42-4e63-a951-147f1781ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.255411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e7513f4e-0b42-4e63-a951-147f1781ef97 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:40.256648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e7513f4e-0b42-4e63-a951-147f1781ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:40.266596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:40.422882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c8038-b744-4f01-94ec-3d150948e0be None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:40.439326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c8038-b744-4f01-94ec-3d150948e0be None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:40.445517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:40.446165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:40.446450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:40.446727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:40.456903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:40.458714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23069f63-554f-42bd-adf6-48e7f3df4997 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:40.469976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 748/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:53:41.168286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cac61-2954-4ea4-a0fb-e62b0c770a05 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:41.168513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cac61-2954-4ea4-a0fb-e62b0c770a05 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:41.168728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cac61-2954-4ea4-a0fb-e62b0c770a05 None None] PATH_INFO: `/v3` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:41.170171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 769/1517] 10.222.0.70 () {54 vars in 925 bytes} [Tue Aug 30 00:53:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:41.177105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9b95-3245-4278-948f-67b1ffe2b24c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:41.177377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9b95-3245-4278-948f-67b1ffe2b24c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:41.177646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9b95-3245-4278-948f-67b1ffe2b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:41.245313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-790f9b95-3245-4278-948f-67b1ffe2b24c None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:41.246820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-790f9b95-3245-4278-948f-67b1ffe2b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:41.267205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:41.330307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 749/1518] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Aug 30 00:53:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:45.233813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2bda-73d4-4898-bf38-22959a574477 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:45.234743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2bda-73d4-4898-bf38-22959a574477 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:45.234743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2bda-73d4-4898-bf38-22959a574477 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:45.236864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 770/1519] 10.222.0.17 () {54 vars in 961 bytes} [Tue Aug 30 00:53:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:45.244901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cc8a1-23e2-427d-b1fa-b9032cae91f3 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:45.245364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cc8a1-23e2-427d-b1fa-b9032cae91f3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:45.245478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cc8a1-23e2-427d-b1fa-b9032cae91f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:45.266043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e0cc8a1-23e2-427d-b1fa-b9032cae91f3 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:45.267268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e0cc8a1-23e2-427d-b1fa-b9032cae91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:45.276890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 750/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Aug 30 00:53:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:53:45.288258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d2bda-73d4-4898-bf38-22959a574477 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:45.304451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d2bda-73d4-4898-bf38-22959a574477 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:45.310108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:45.310701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:45.310976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:45.311271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:45.319770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:45.321355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37b7e3-0ee0-4241-8b31-23f20b7bf22e service placement] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:45.331485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 771/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:51.688684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4065ba-aa27-4530-b9ce-6034490e639a None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:51.689791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4065ba-aa27-4530-b9ce-6034490e639a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:51.689791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4065ba-aa27-4530-b9ce-6034490e639a None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:51.691655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:53:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:51.700581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba3368-dfc0-4504-bd77-d5f6b97f6cf3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:51.701284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba3368-dfc0-4504-bd77-d5f6b97f6cf3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:51.701388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba3368-dfc0-4504-bd77-d5f6b97f6cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:51.728061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e0ba3368-dfc0-4504-bd77-d5f6b97f6cf3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:51.729945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e0ba3368-dfc0-4504-bd77-d5f6b97f6cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:51.740098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:51.901861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4065ba-aa27-4530-b9ce-6034490e639a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:51.917456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4065ba-aa27-4530-b9ce-6034490e639a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:51.923490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:51.924090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:51.924367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:51.924640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:51.932910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:51.934506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b1883-de13-459b-a7f1-ae70388381e6 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:51.944343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 752/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:52.811688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990eeb2-9233-4732-9ec1-c75828fdabb8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:52.812248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990eeb2-9233-4732-9ec1-c75828fdabb8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:52.812248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990eeb2-9233-4732-9ec1-c75828fdabb8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:52.813560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 00:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:52.819429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999af0-6eef-49a7-a343-48fed1f76a6d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:52.820139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999af0-6eef-49a7-a343-48fed1f76a6d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:52.820243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999af0-6eef-49a7-a343-48fed1f76a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:52.846820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1b999af0-6eef-49a7-a343-48fed1f76a6d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:52.848092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1b999af0-6eef-49a7-a343-48fed1f76a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:52.856682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 00:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:53.013678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990eeb2-9233-4732-9ec1-c75828fdabb8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:53.037256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990eeb2-9233-4732-9ec1-c75828fdabb8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:53.046113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:53.046716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:53.046991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:53.047285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:53.056379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:53.058023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82ca55-c076-4f79-b017-6ddc196c6813 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:53.068961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 774/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:57.263971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:57.264624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:57.264624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:57.266684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:53:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:57.273030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917d4d-6bdf-4e2d-b233-7917e5d8aee6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:57.273596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917d4d-6bdf-4e2d-b233-7917e5d8aee6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:57.273801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917d4d-6bdf-4e2d-b233-7917e5d8aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:57.300600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3f917d4d-6bdf-4e2d-b233-7917e5d8aee6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:57.302094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3f917d4d-6bdf-4e2d-b233-7917e5d8aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:57.314241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:53:57.628821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:57.630669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:57.753963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dacd2-a9f8-4b5a-b5f3-11defdbc0f89 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:57.760070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:57.760692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:57.761149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:57.761670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:57.775333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:57.776824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb147-772d-4e4b-9453-04b6b23d237a service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:57.787751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 755/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:53:59.744370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:59.744908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:59.745024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:59.747112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:53:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:59.756973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcf3d-a7e9-4b0c-9263-fda7b580ef33 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:59.757426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcf3d-a7e9-4b0c-9263-fda7b580ef33 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:59.757530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcf3d-a7e9-4b0c-9263-fda7b580ef33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:59.781195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9d3fcf3d-a7e9-4b0c-9263-fda7b580ef33 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:59.782447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9d3fcf3d-a7e9-4b0c-9263-fda7b580ef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:59.792206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:00.095260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:00.098205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:00.116866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aa61d-b842-4a43-a4fc-93d8e23fe54c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:00.121370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:00.121972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:00.122241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:00.122555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:00.131948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:00.133402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac580-b73d-4504-bced-206af666ddd9 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:00.142884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 777/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:04.045910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:04.046479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:04.046479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:04.047866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:54:04.054039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca66b2-b4a0-4e7a-9537-bc602699eded None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:04.054522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca66b2-b4a0-4e7a-9537-bc602699eded None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:04.054651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca66b2-b4a0-4e7a-9537-bc602699eded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:04.077520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ecca66b2-b4a0-4e7a-9537-bc602699eded None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:04.078741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ecca66b2-b4a0-4e7a-9537-bc602699eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:04.086796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:04.645197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:04.647288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:04.718308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:04.732477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c5352-68de-4ec1-808e-1510c580decf None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:04.736441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:04.737054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:04.737366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:04.737599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:04.746652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:04.748192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d97f23-823b-4d72-8e2c-deb64a7d5434 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:04.757564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 758/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:06.687695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:06.688290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:06.688290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:06.689528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 00:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:54:06.694972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a39e-54d5-4a6e-9dc6-3d06d47f36bd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:06.695492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a39e-54d5-4a6e-9dc6-3d06d47f36bd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:06.695492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8a39e-54d5-4a6e-9dc6-3d06d47f36bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:06.717733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6be8a39e-54d5-4a6e-9dc6-3d06d47f36bd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:54:06.718924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6be8a39e-54d5-4a6e-9dc6-3d06d47f36bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:06.728193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 00:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:54:07.283655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:54:07.286979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:54:07.304879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb0b0-87c0-48b2-9ed1-9191efa5ba6e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:54:07.310384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:54:07.311033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:54:07.311311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:54:07.311583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:54:07.321988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:54:07.324033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9549f-3e55-41b6-9bbe-5fc9b8c94a1a service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:54:07.334770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 780/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:54:16.711847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=11 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:54:17.330504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:32.865942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd6c8-1df2-43d5-91f7-a7b616059745 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:32.867260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd6c8-1df2-43d5-91f7-a7b616059745 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:32.867766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd6c8-1df2-43d5-91f7-a7b616059745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:32.894131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a05dd6c8-1df2-43d5-91f7-a7b616059745 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:32.895979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a05dd6c8-1df2-43d5-91f7-a7b616059745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:32.896905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098632-2af9-431b-9362-2521d76eebf7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:32.897415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098632-2af9-431b-9362-2521d76eebf7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:32.897415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31098632-2af9-431b-9362-2521d76eebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:32.908936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 760/1540] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:32.925970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:32.926603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:32.927138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:32.935558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-31098632-2af9-431b-9362-2521d76eebf7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:32.944166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-31098632-2af9-431b-9362-2521d76eebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:32.959276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 781/1541] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:32.967924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:32.969137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bed3-f907-437e-b5cd-036a8ce9fb8b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:32.969420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bed3-f907-437e-b5cd-036a8ce9fb8b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:32.969689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bed3-f907-437e-b5cd-036a8ce9fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:32.970311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:32.996712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 761/1542] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:33.000131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f881bed3-f907-437e-b5cd-036a8ce9fb8b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.001447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f881bed3-f907-437e-b5cd-036a8ce9fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.002607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.022521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 782/1543] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:33.024646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.024938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.025209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.026589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713eec-8c52-4a3b-912d-f5f816ef14d0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.046290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.047130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.047511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.047844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.048728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.049076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.052401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fa3a7-4d1f-49f5-8fb0-00b7be62ce10 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.053442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.055058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.072120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 783/1544] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.073529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.083834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 762/1545] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:55:33.090731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a7b9-3e0b-4400-84fc-b3ce9dfc2b9d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.091120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a7b9-3e0b-4400-84fc-b3ce9dfc2b9d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.091310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8a7b9-3e0b-4400-84fc-b3ce9dfc2b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.096664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b66d2-8a46-49fd-9e44-77c5725ef099 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.111806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.112440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.112712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.112976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.113644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.119358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.120339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:33.120339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:55:33.120339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:55:33.121239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.138946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a2a8a7b9-3e0b-4400-84fc-b3ce9dfc2b9d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.147852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a2a8a7b9-3e0b-4400-84fc-b3ce9dfc2b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.156733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 784/1546] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:33.157618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.174572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 763/1547] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.175940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.176216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.176489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.188296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f3a7-9d72-45e9-97c8-cc6a029a2026 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.209522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.210165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.210893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.211186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.211829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.213983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.234773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.236020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.267096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 764/1548] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.268023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.285383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ad77-4431-41f8-b6c4-4ef9c52d78ac None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.294398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.295046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.295377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.295643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.296290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.296587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.298244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbabff-237f-4acf-b704-ca18e0191c7d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.301272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 785/1549] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.302047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.315449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 765/1550] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:33.316888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ffcd-f92c-4fa0-a9be-d4c24477e839 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.317177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ffcd-f92c-4fa0-a9be-d4c24477e839 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.317452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ffcd-f92c-4fa0-a9be-d4c24477e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.348081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-09e6ffcd-f92c-4fa0-a9be-d4c24477e839 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.349353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-09e6ffcd-f92c-4fa0-a9be-d4c24477e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.381291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ef3-2194-428f-8681-cdad7b3820ab admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.386377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.387030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.387305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.387575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.388213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.390198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6847126-863f-438e-8372-688a6a6145a7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.422687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 766/1551] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.423997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.424283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.424557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.430636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 786/1552] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.431167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.431463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.431766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.445829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.447072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.453146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.454338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.505926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 767/1553] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.507048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.516752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46063e-395c-431a-b61e-add9e6402437 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.521363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.521596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 787/1554] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.521961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.522237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.522423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.522535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.523371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.525350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419914e2-3c69-4028-9233-3d5db009a3bc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.534862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1396-d0f9-4927-a988-4aef0af95302 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.540901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.541518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.541822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.542092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.542714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.544989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068288-a3aa-46c9-9870-493a283aea46 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.548532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 768/1555] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.549772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98d123-87b3-405c-b82e-43c4770f6136 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.550058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98d123-87b3-405c-b82e-43c4770f6136 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.550326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98d123-87b3-405c-b82e-43c4770f6136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.569422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:33.570408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-df98d123-87b3-405c-b82e-43c4770f6136 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.571650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-df98d123-87b3-405c-b82e-43c4770f6136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.577074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 788/1556] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.587157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 769/1557] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.587467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.587467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.587667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.587768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.587853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.587853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.605525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.606018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.606761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.607414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.629439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:33.673303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 789/1558] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.673516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 770/1559] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.677743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.677908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.687263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e65c6b-89c7-4fde-9ca9-4af5ab58473b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.688336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749593e-7dac-4f35-b851-5ac4d42c61ba None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.692576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.692947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.693190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.693467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.693576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.693735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.693830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.694082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.694368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.695409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.695738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.695915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:33.695915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:55:33.695915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:55:33.697124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.697338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f076-72ac-44d2-a1a5-83709cf65689 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.711778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 771/1560] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:33.713199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.713571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.713851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.731157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 790/1561] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.731157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.735148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.736367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.739364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d384a56-4b29-461c-b4f3-2bcb22e854d4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.744525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.745131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.745244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 772/1562] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.749135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.749135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.749316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.749406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.749406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db9235-6f50-4fae-93be-48a3246b4376 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.759461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e47f8-3ba2-48f4-bea9-b9aacbd28a8f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.759782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 791/1563] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:33.761012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.761297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.761567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.766594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.767237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.767521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.767796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.768427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.770366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15745bd-f280-4daa-a3ba-b553ec1cc94e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.782059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.783385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.792982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 792/1564] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.796495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 773/1565] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.796619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.796737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.796737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.796737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.807198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca344fd8-1e5b-4885-9cbd-2e8156a0412f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.812604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.813213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.813494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.813762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.814403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.814701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.815819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:33.816337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.817037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:33.826154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 774/1566] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:33.826935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.829117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 793/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:33.829942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.841051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c465-7d34-4255-96dc-295e4986526e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.841900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354728f6-4092-4ee3-988e-f08733e303e4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.845650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.846251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.846529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.846699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.846808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.847351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.847464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.847626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.848015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.848654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.849190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.849781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:33.849781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:55:33.849781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:55:33.850959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.861235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 794/1568] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:33.862070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.872516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ec34-a868-4d6c-8c8a-01ed6e4687a5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.877249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.877842 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.878112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.878400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.879059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:33.879360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.880505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 775/1569] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:33.881012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.881352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.899445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 795/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:33.899445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.915520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c981d-06c3-449c-b7f2-8147b81684bc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.962821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a060b-5b1b-4463-8ad2-469e4e06466d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.968976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.969596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.969880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.970181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.970831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.972669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.981857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:33.982482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:33.982759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:33.983069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:33.983716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:33.986957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 776/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:33.986957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:33.987217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:33.995867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999f5f-30bb-41d5-84cf-5bd07335b09e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:33.996431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 796/1572] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:33.997223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.001028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.001625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.002038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.002165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.002936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:34.003274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.004854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.007894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afbe1e-fad7-4a0f-a360-de55585b47d5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.012700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.013441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.013716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.013977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.014603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.016516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.017485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 777/1573] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:34.018431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.028571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6b0f5-49b5-4e12-bbf9-9f47d6ecabbd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.033182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.033778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.034052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.034323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.034985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:34.035310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.036892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.050359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 778/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:34.054061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 797/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:33 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.054279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.054568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.061971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.065542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb4ad8-6ca4-406d-bc1e-9df087cca11d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.094757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.123648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779556d-e7d6-4eca-97b9-2577a8596466 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.127844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.128518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.128678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.128801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.129075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.129294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.129578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.129676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.129863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.130483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.131325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.132593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.142247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 779/1576] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.142856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.153849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790bb6-3b88-4858-b280-7386df57c382 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.157970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 798/1577] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.157970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.161756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.162366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.162640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.162909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.163943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.165371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea5181-854f-4298-b8f6-c6e89eb32498 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.166331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.170122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.170762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.171067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.171340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] PATH_INFO: `/v3/projects/e36d98f444ec4d12a1a55ef450de63a5/users/f2a8b56e48b445709054849f4893767a/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.186273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36d98f444ec4d12a1a55ef450de63a5, user_id=f2a8b56e48b445709054849f4893767a, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.188992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.194312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 780/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.195617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.207479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 799/1579] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:34 2022] PUT /identity/v3/projects/e36d98f444ec4d12a1a55ef450de63a5/users/f2a8b56e48b445709054849f4893767a/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:34.208281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.262937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12825a2-621a-4905-9203-fc1d2d64900d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.267736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.268351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.268628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.268899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.269569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.271248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.271785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d136c-ca81-40e7-980f-5673474e0d7a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.277477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.278093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.278388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.278640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.279296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.281206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.281763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 781/1580] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.282634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.292917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0697dbc-ef61-41ab-bd58-9586f323c8d1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.297783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.298553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.298974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.299311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.300086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.302500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.312264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 800/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.312264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.328953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 782/1582] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.328953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.336805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c84317-5f90-4d25-a58f-5a70d5918d89 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.369178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.382104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52cedb8-6dc5-42f4-ac8f-a895a3a042f8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.387667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.388452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.388803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.389142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.389851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.391880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.396283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.396967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.397295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.397578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] PATH_INFO: `/v3/projects/1ab0b517d41b4b83af9af6f1fcce13d7/users/e0e5c1345e4049f0bf5fc17ef71b7133/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.416239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab0b517d41b4b83af9af6f1fcce13d7, user_id=e0e5c1345e4049f0bf5fc17ef71b7133, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.417058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:34.417058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:55:34.417058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:55:34.419063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.420279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 801/1583] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.421306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.437828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 783/1584] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:34 2022] PUT /identity/v3/projects/1ab0b517d41b4b83af9af6f1fcce13d7/users/e0e5c1345e4049f0bf5fc17ef71b7133/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:34.438870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.447860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.487089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047135b-dc55-4d64-9718-641b41da60a8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.506627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e62e-870c-4454-a415-bbd45576ead2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.511906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.512524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.512801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.513074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.513737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.515703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d34d-d7e3-4a20-9be5-1695bf7600ef admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.539500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 784/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.540879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.541302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.541450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.546586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.547352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.547726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.548094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.548827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.550494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.560633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:34.561230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 802/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.561837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:34.562012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.571038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 785/1587] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:34.571973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3cdaa-47f1-4140-affb-49ccdf5aed21 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.575259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.576972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.577632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.577908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.578176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] PATH_INFO: `/v3/projects/7e919c196d074b8989b4719267ce24f8/users/22f11e29c4ad4712a54877b0895d28a4/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.584934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2ab49-f604-4604-9c83-732dcb87c5dc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.590194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.590770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.591072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.591345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.592004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.593883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.594102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e919c196d074b8989b4719267ce24f8, user_id=22f11e29c4ad4712a54877b0895d28a4, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.596847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.611283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 803/1588] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:34 2022] PUT /identity/v3/projects/7e919c196d074b8989b4719267ce24f8/users/22f11e29c4ad4712a54877b0895d28a4/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:34.615095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 786/1589] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.615880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.618795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.625414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec76c0-34f3-4661-9747-ceedeed0f4c6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.675568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.679382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929286e-065b-408b-98ad-f8db5df1541e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.682161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.682792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.683106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.683368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.684001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.685473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.685918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.686116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.686394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.686667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.687354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.688954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963836c6-19d1-4e04-b6b9-a6e9e34607cc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.700150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 804/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.701551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.701840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.702113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.711618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 787/1591] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:34.712533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.722362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8857ca-b0e6-4eeb-8ea6-e1a7f1b73b54 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.723757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:34.724982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:34.728153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.728781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.729062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.729336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.730007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.731868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.734505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 805/1592] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:34.737982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.744239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 788/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.746806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58a569-6720-42ec-9bc1-7fc642e86f12 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.748096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.751796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.752415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.752789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.752959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.753615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.755273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e1476-0c9b-4d43-a14d-8554de78747c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.767170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 806/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.768718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.768718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.768964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.790826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:34.792068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:34.800182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 807/1595] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:34.800956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.810135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:34.810555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff4a1e-06df-446a-9747-a689e1b6bf60 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.814466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab570a-b0d8-47a3-8282-9546494e177d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.817760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.818556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.818829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.819127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.820214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.820462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.821435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.821850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.822061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.822579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.832480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.832623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 808/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.833226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.833924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.843796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 789/1597] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:34.844472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63651c-9afb-4078-9f1b-2c6ebbfc3be5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.844751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.849123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.849718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.849990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.850250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.850907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.853045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.854105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff17e2-14ee-42b6-97f9-5369b20915e4 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.858876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.859514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.859789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.860078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.860856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.862270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.864880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 809/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.865796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.873917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 790/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.875059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.879180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0fa0e-1b06-4cef-8c22-7cee009849cb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.883647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.884416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.884787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.885153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.885645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d909a8-7f8a-47ef-ba96-07338d3221df admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.890816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.891460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.891737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.892131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.892645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.894319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.895656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.897102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.911128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 791/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.911335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 810/1601] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:34.911335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.911878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.923201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0923454-3257-4ccf-b783-d11cafbd080a service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.929089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.930107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.930482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.931225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.934695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44751b0a-699e-4ab4-a290-55bc1a7be964 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.944269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.946427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.946638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.947330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.947660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.947946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] PATH_INFO: `/v3/projects/9a1cf0a235aa4669a9276c293a78f2eb/users/1d1c13089bd14d48ba637468182eee14/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.959935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 811/1602] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:34.961347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:34.974487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5590e0-a8ec-4ac1-8702-a8f7726f4fa0 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:34.978487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1cf0a235aa4669a9276c293a78f2eb, user_id=1d1c13089bd14d48ba637468182eee14, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.980371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:34.980988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:34.981262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:34.981538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:34.982180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:34.983935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.984084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:34.998228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 812/1603] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:34.998752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.011893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 792/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:34 2022] PUT /identity/v3/projects/9a1cf0a235aa4669a9276c293a78f2eb/users/1d1c13089bd14d48ba637468182eee14/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:35.011893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.012209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b61ce-7393-4fad-8380-a94d6e8bd3ac admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.081841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.082995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.083140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.083477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] PATH_INFO: `/v3/projects/1da2a1cc47754a40a0ebf71905bb43ed/users/b238cd707f7c4da0958c9a48f2fd947a/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.090113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032a9a-d6a4-4f82-be17-ed3165555b64 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.095331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.096054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.096403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.096734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.097496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.099214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.101133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da2a1cc47754a40a0ebf71905bb43ed, user_id=b238cd707f7c4da0958c9a48f2fd947a, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.104145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.115592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 793/1605] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:35.116670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.124720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 813/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:34 2022] PUT /identity/v3/projects/1da2a1cc47754a40a0ebf71905bb43ed/users/b238cd707f7c4da0958c9a48f2fd947a/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:35.128131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4d4-08da-4845-b08f-5f11c8b6888d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.129119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.196554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.197602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.198166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.198501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] PATH_INFO: `/v3/projects/bd5812ff7859449882b0e82e3b9bb38f/users/4743224ed4eb406aab59e018d69a3116/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.209036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:35.218170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5812ff7859449882b0e82e3b9bb38f, user_id=4743224ed4eb406aab59e018d69a3116, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.221055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4be81f-a982-45e1-afa4-c7a5bbab24d1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.223951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d874-9e79-4d12-8ea2-a7bf4b17a9d3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.232717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.235747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.236068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.236626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] PATH_INFO: `/v3/projects/8809a2a9708d415f996c33fc88edcd82/users/0dfe54df1c2b45f3aba25b8340b60797/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.244341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 794/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:35 2022] PUT /identity/v3/projects/bd5812ff7859449882b0e82e3b9bb38f/users/4743224ed4eb406aab59e018d69a3116/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:35.246028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3159e-b569-429e-88f4-fc36dab17073 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.246253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3159e-b569-429e-88f4-fc36dab17073 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.246569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3159e-b569-429e-88f4-fc36dab17073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.264261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8809a2a9708d415f996c33fc88edcd82, user_id=0dfe54df1c2b45f3aba25b8340b60797, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.268574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.286312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-03b3159e-b569-429e-88f4-fc36dab17073 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.287814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-03b3159e-b569-429e-88f4-fc36dab17073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.290287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 814/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:35 2022] PUT /identity/v3/projects/8809a2a9708d415f996c33fc88edcd82/users/0dfe54df1c2b45f3aba25b8340b60797/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:35.291894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.323447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:35.384812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 795/1609] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:35.386751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7090-7b64-45f3-9eea-2fb765ea7f6f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.387059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7090-7b64-45f3-9eea-2fb765ea7f6f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.387336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7090-7b64-45f3-9eea-2fb765ea7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.400240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b89b7a-13a0-4e16-b640-ab9e5d6af309 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.407154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.408508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.408785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.409047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] PATH_INFO: `/v3/projects/79940002af7f4460993cba3e3b4b15de/users/380be9f89d3e4dfb9e5645930ced10a4/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.441207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79940002af7f4460993cba3e3b4b15de, user_id=380be9f89d3e4dfb9e5645930ced10a4, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.445810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07b9ee-c28f-4a4f-8ec0-18f468087a5a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.474129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 815/1610] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:35 2022] PUT /identity/v3/projects/79940002af7f4460993cba3e3b4b15de/users/380be9f89d3e4dfb9e5645930ced10a4/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:35.476013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.476382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.476697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.490032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5f4f7090-7b64-45f3-9eea-2fb765ea7f6f None None] MFA Rules not processed for user `b238cd707f7c4da0958c9a48f2fd947a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.491433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5f4f7090-7b64-45f3-9eea-2fb765ea7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.529740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] MFA Rules not processed for user `4743224ed4eb406aab59e018d69a3116`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.531263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.593553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 796/1611] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:35.598018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54510975-e50b-4b0a-a2fc-49c408ac2735 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.598018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54510975-e50b-4b0a-a2fc-49c408ac2735 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.598018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54510975-e50b-4b0a-a2fc-49c408ac2735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.663916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 816/1612] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:35.663916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:35.683183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-54510975-e50b-4b0a-a2fc-49c408ac2735 None None] MFA Rules not processed for user `0dfe54df1c2b45f3aba25b8340b60797`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.686159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-54510975-e50b-4b0a-a2fc-49c408ac2735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.734683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:35.741511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d14a57-3452-4fd0-aca6-d986973d0baf None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:35.747629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:35.748331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.748603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.748875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.808737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 797/1613] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:35.816588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052a87-ecd2-4bbf-9f16-2b8886c1d92c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.817162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052a87-ecd2-4bbf-9f16-2b8886c1d92c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.817696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052a87-ecd2-4bbf-9f16-2b8886c1d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.849075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:35.851583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc52f5-871a-43f5-8f0a-17b94cf08586 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:35.865169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9e052a87-ecd2-4bbf-9f16-2b8886c1d92c None None] MFA Rules not processed for user `380be9f89d3e4dfb9e5645930ced10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.869061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9e052a87-ecd2-4bbf-9f16-2b8886c1d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:35.870191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 817/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:35.872943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3468b9-ea2b-4ce2-a597-75555eb5566c None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.873544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3468b9-ea2b-4ce2-a597-75555eb5566c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.874112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3468b9-ea2b-4ce2-a597-75555eb5566c None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.883686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 818/1615] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 30 00:55:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:35.885930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ef6a-20ad-4ea3-8fe4-da3d90591d2c None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.886416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ef6a-20ad-4ea3-8fe4-da3d90591d2c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.886836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ef6a-20ad-4ea3-8fe4-da3d90591d2c None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.894413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 819/1616] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 30 00:55:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:35.896139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:35.896465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:35.896698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:35.919702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:35.957156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:35.958656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:36.004893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 798/1617] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:36.011146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.012928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.013596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.051810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 820/1618] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 30 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:36.052701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.065711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832533a7-05e0-4e21-a59b-432bbbf48904 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.071409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.072067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.072352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.072625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.073339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.075328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.077692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:36.080650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:36.105139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 799/1619] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 30 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:36.116405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.118417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 821/1620] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:36 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:36.119450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.145499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e669280-a97f-4768-b072-581194fe07f1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.204353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:36.228490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fd2fe-d5ba-4f77-a42f-8e27491631d6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.244381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.245307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.246875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.246875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.300853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.301492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.301771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.302044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.365155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.368149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.385182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 822/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:36.390714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.393092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.399097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.410830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 800/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:36.615457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b292efb-14d8-48af-b032-09d221a2ea1f service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.627512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.630584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.630584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.630584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.630584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.635683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.684719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 823/1623] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:36 2022] POST /identity/v3/users => generated 290 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:36.688302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.706224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab709-0ec6-4ef2-9684-797715e95f4b service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.716263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.717056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.719866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.720263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.721405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.723925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.754733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 801/1624] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:36.760071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.786163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563722-65cc-4788-9fd7-ef8e5f38963e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.796902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:36.797850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.798610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.803660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] PATH_INFO: `/v3/projects/41f16ae5bf064c14b178fb372b9edf2d/users/f7032316aef64d33b00b2b2fa736001f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.843426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f16ae5bf064c14b178fb372b9edf2d, user_id=f7032316aef64d33b00b2b2fa736001f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:36.846547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a38423-89ad-4a42-b81c-c9a30fd2bde3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:36.882804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 824/1625] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:36 2022] PUT /identity/v3/projects/41f16ae5bf064c14b178fb372b9edf2d/users/f7032316aef64d33b00b2b2fa736001f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:36.886670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:36.902872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2f674-3882-4efe-ace8-75ec7ec590e5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:36.959557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1cac-9a9f-4607-af44-a85e7bccd309 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.961216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1cac-9a9f-4607-af44-a85e7bccd309 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.961944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f1cac-9a9f-4607-af44-a85e7bccd309 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.963880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 825/1626] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 30 00:55:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:55:36.972303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161872-f613-41d6-b01a-19567485fa43 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:36.974827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161872-f613-41d6-b01a-19567485fa43 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:36.975361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161872-f613-41d6-b01a-19567485fa43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:36.997911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:37.016240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-0c161872-f613-41d6-b01a-19567485fa43 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:37.019688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0c161872-f613-41d6-b01a-19567485fa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.022285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:37.023957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.024251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.024538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.026093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:37.030063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f24a-1f3d-4d2d-a7ed-a20587c15f86 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:37.057758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 802/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:37.059401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002dd80-1074-4064-b37f-0fe80c76279e None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.060842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002dd80-1074-4064-b37f-0fe80c76279e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.062032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002dd80-1074-4064-b37f-0fe80c76279e None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.063435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 803/1628] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 30 00:55:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:37.066634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002dd80-1074-4064-b37f-0fe80c76279e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:37.098610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002dd80-1074-4064-b37f-0fe80c76279e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:37.111156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:37.113059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.113348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.113778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] PATH_INFO: `/v3/projects/41f16ae5bf064c14b178fb372b9edf2d/users/f7032316aef64d33b00b2b2fa736001f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.119162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f16ae5bf064c14b178fb372b9edf2d, user_id=f7032316aef64d33b00b2b2fa736001f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:37.126722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b470e8-0019-4aa3-a37c-5325eb71b217 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:37.149099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:37.160747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 804/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:37 2022] PUT /identity/v3/projects/41f16ae5bf064c14b178fb372b9edf2d/users/f7032316aef64d33b00b2b2fa736001f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:37.164199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.165137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.165427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.200804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 826/1630] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:37.203794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.204086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.204365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.223698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:37.226104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.276434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] MFA Rules not processed for user `f7032316aef64d33b00b2b2fa736001f`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:37.279858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:37.423118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 805/1631] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:55:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:37.424489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:37.471108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 827/1632] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:37.478325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:37.501947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883300d-cfab-4acb-a65f-303dda7533d3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:37.507406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:37.508171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.508581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.508977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.524230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:37.526213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:37.542090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 806/1633] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:37.593524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbfa8-9f07-4b08-ae92-cbcb02829459 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:37.606551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:37.607584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:37.611788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:37.612316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:37.647414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:37.649629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:37.687443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 828/1634] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:38.371223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:38.391630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b50762-c4c9-4c14-a740-1421cabfe0f9 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:38.405487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:38.406444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:38.406882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:38.407328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:38.422978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:38.425658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:38.447496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 807/1635] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:38.630776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:38.652261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4582-bc4e-4ab3-92ea-25b72c3e3c15 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:38.667891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:38.668519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:38.668790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:38.671922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:38.700735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:38.715929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a966e-b52c-44fe-a7b0-fb8f042cea10 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:38.730182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:38.730822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:38.731318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:38.731591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:38.845460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:38.848490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:38.850471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:38.877557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:38.877802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 808/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:38.887578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cff4b-17c0-4a7d-99e5-a4005f6263e4 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:38.925372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 829/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:39.376271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:39.527717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de21d-6f7d-40da-a737-2ddac7689a0c service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:39.538523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:39.539726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:39.540228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:39.540714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] PATH_INFO: `/v3/users/0dfe54df1c2b45f3aba25b8340b60797` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:39.542305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfe54df1c2b45f3aba25b8340b60797)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:39.543814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: WARNING py.warnings [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:39.543814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: warnings.warn(msg) Aug 30 00:55:39.543814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: Aug 30 00:55:39.545380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:39.693299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfe54df1c2b45f3aba25b8340b60797'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:39.729423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfe54df1c2b45f3aba25b8340b60797'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:39.745398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfe54df1c2b45f3aba25b8340b60797'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:39.746065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfe54df1c2b45f3aba25b8340b60797'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:39.790444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfe54df1c2b45f3aba25b8340b60797'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:39.799172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 809/1638] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:39 2022] DELETE /identity/v3/users/0dfe54df1c2b45f3aba25b8340b60797 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:39.803936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:39.805325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:39.805828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:39.844383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:39.845669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:39.859060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:39.941292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 830/1639] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:39.965279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:39.967675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:40.080845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac4d44-45b1-4a2f-8db9-f2e1d0d95c42 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:40.086951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:40.087633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:40.087918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:40.093397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:40.157280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:40.165137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf2f300-a787-475b-bf95-2dc2261a2577 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:40.201324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 810/1640] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:41.011811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:41.015624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:41.043218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e70d78-f697-4d17-9da2-c88835d6d979 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:41.049616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:41.050818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:41.051363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:41.051873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] PATH_INFO: `/v3/users/b238cd707f7c4da0958c9a48f2fd947a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:41.053730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b238cd707f7c4da0958c9a48f2fd947a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:41.057826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:41.278230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b238cd707f7c4da0958c9a48f2fd947a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:41.304903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b238cd707f7c4da0958c9a48f2fd947a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:41.319871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b238cd707f7c4da0958c9a48f2fd947a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:41.374912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b238cd707f7c4da0958c9a48f2fd947a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:41.418973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b238cd707f7c4da0958c9a48f2fd947a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:41.449845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 831/1641] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:41 2022] DELETE /identity/v3/users/b238cd707f7c4da0958c9a48f2fd947a => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:41.455677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:41.456156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:41.456263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:41.501428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:41.502680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:41.541536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:41.675169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 811/1642] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:41.717564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:41.722544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:41.869417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71615fa7-2108-4043-b0cb-9f4c8a4aa47d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:41.873882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:41.874670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:41.875060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:41.875729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:41.900500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:41.902775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66338-92c5-49e3-83bf-45a25862b315 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:41.922298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 832/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:43.094524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:43.096099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:43.116148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c542-6c26-4a24-98fc-6599d8563bbc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:43.140768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:43.141843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.142606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.144251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] PATH_INFO: `/v3/projects/8809a2a9708d415f996c33fc88edcd82` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.148822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8809a2a9708d415f996c33fc88edcd82)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:43.154846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:43.259179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8809a2a9708d415f996c33fc88edcd82', 'request_id': 'req-e38d5f55-a82d-441b-88b2-a1287be2755c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:43.268942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8809a2a9708d415f996c33fc88edcd82', 'request_id': 'req-e38d5f55-a82d-441b-88b2-a1287be2755c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:43.292151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8809a2a9708d415f996c33fc88edcd82', 'request_id': 'req-e38d5f55-a82d-441b-88b2-a1287be2755c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:43.324122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] The token cache is being invalidate because project 8809a2a9708d415f996c33fc88edcd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:43.324728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:43.325804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e38d5f55-a82d-441b-88b2-a1287be2755c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:43.327973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 812/1644] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:43 2022] DELETE /identity/v3/projects/8809a2a9708d415f996c33fc88edcd82 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:43.332673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.333222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.333222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.380939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:43.382235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:43.457964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:43.532763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 833/1645] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:43.540852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.541243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.541659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.614166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:43.618235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:43.646365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 813/1646] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:43.647756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:43.666839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd57cc2-e382-4a97-a26a-89956f87e605 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:43.678708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:43.680900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.682228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.682726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.684332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:43.686005 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:43.691292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:43.736355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:43.746748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 834/1647] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:43.772493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ea95f-04e2-48c7-b2ee-c759d6b362c2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:43.786992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:43.788864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:43.788864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:43.790945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:43.795967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:43.805047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:43.868136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 814/1648] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:43 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:55:43.869667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:43.880901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fd85-7304-4d69-b1d7-3e2366d0bbe1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.106208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.107312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.107926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.108417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.109477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.112737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.146281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 835/1649] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:43 2022] POST /identity/v3/users => generated 290 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:44.151076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:44.166514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7731f-aac3-446c-ac50-b3637a96bb79 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.181071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.182202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.182723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.183341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.184263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.190179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.220780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 815/1650] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:44.225990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:44.249739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:44.252788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:44.253922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12162db-fd27-4e85-88ef-843d511c712f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.296328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.302467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.302910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.303296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.310070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.317607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.326241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc466bb0-6ff3-4a49-a61c-21b8678d57d9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.337047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.339792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.340283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.340713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] PATH_INFO: `/v3/projects/1da2a1cc47754a40a0ebf71905bb43ed` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.342680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da2a1cc47754a40a0ebf71905bb43ed)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.347590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.369119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 836/1651] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:44.373555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:44.427785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd70e3-2f02-4e73-a2eb-203438d6a8cf admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.461698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.463904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.468809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.469600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] PATH_INFO: `/v3/projects/95a75c155ecb4895a57e8824a4a7f1e3/users/c1421170837d4168878252e3b2a2fae6/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.536377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a75c155ecb4895a57e8824a4a7f1e3, user_id=c1421170837d4168878252e3b2a2fae6, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.548742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.552490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da2a1cc47754a40a0ebf71905bb43ed', 'request_id': 'req-59d86881-0586-4581-8a16-2f84bb7cae85'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.565427 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:44.567211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da2a1cc47754a40a0ebf71905bb43ed', 'request_id': 'req-59d86881-0586-4581-8a16-2f84bb7cae85'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.593782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 837/1652] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:44 2022] PUT /identity/v3/projects/95a75c155ecb4895a57e8824a4a7f1e3/users/c1421170837d4168878252e3b2a2fae6/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:44.606933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:44.609696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:44.614823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da2a1cc47754a40a0ebf71905bb43ed', 'request_id': 'req-59d86881-0586-4581-8a16-2f84bb7cae85'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.639142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] The token cache is being invalidate because project 1da2a1cc47754a40a0ebf71905bb43ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:44.639567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.640824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-59d86881-0586-4581-8a16-2f84bb7cae85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.644947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 816/1653] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:44 2022] DELETE /identity/v3/projects/1da2a1cc47754a40a0ebf71905bb43ed => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:44.648863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ccc77-d770-419b-8360-3b595510957d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.648863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ccc77-d770-419b-8360-3b595510957d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.648863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ccc77-d770-419b-8360-3b595510957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.754488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-616ccc77-d770-419b-8360-3b595510957d None None] MFA Rules not processed for user `c1421170837d4168878252e3b2a2fae6`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:44.755990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-616ccc77-d770-419b-8360-3b595510957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:44.829402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d270e8d-b462-4133-bdd3-6410100b7c9d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:44.842460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:44.843220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.843355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.843600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] PATH_INFO: `/v3/users/380be9f89d3e4dfb9e5645930ced10a4` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.847654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=380be9f89d3e4dfb9e5645930ced10a4)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:44.852557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:44.894282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:44.937537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 817/1654] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:44.942132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:44.942428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:44.947265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:44.984476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '380be9f89d3e4dfb9e5645930ced10a4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:44.996777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:45.002354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:45.004938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '380be9f89d3e4dfb9e5645930ced10a4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:45.005929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '380be9f89d3e4dfb9e5645930ced10a4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:45.031287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 818/1655] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:45.031287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.031934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:45.042408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '380be9f89d3e4dfb9e5645930ced10a4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:45.053783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ee6c-1f46-4a4e-aec1-c841ae629611 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.056777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-090563ea-dc9c-475a-b7d4-4ed29128c07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '380be9f89d3e4dfb9e5645930ced10a4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:45.072634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 838/1656] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:44 2022] DELETE /identity/v3/users/380be9f89d3e4dfb9e5645930ced10a4 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:45.074571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.074867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.075334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.087389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.088044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.088332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.088651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.089417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.091434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d774a3-d952-4524-b2cb-1f9848f2933b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.110434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:45.112841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:45.177811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 819/1657] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:45.180110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.180613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.181726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.240814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 839/1658] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:45.242748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.244336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:45.246744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:45.271701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca5d88-2e6c-42b4-b623-ac2185dc7ce1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.443984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.445215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.446035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.446907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.448134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.450048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.463150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 820/1659] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:45.464053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.486881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e8e7-6bc6-442e-b07b-be9efb555652 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.492497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 840/1660] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:45.496886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.499655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.502719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.503224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.503722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.504593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:45.506260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.510567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.513418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a342b84-e15a-4588-bc14-a13698ade801 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.519286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.519993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.520881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.521169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.521889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.524440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.539280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 821/1661] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:45.542575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.545073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 841/1662] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:45.546967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.561840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eacd9-afae-4d95-8bff-282b53b2cac1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.564541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b48a74-4124-46a3-8819-413106295fad admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.569126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.571655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.571970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.572299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] PATH_INFO: `/v3/projects/6b78673c208546218e5389922cc17f4a/users/2972381ed0074a50bed754c123b0088f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.583087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.583834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.584181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.584646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.585645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.587692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.592431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:45.604145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b78673c208546218e5389922cc17f4a, user_id=2972381ed0074a50bed754c123b0088f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.609061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.637582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 842/1663] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:45 2022] PUT /identity/v3/projects/6b78673c208546218e5389922cc17f4a/users/2972381ed0074a50bed754c123b0088f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:45.642324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.645288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:45.646430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 822/1664] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:45.647899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.665400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:45.767386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937850d9-9f27-4ee0-9690-dfb0d8880ecd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.781325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.782221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.782648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.783090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.783891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.786203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b173a-5421-4eb7-acf9-044408f913f4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.796728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.810976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 823/1665] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:45.812226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.831726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834173a4-542e-4cfa-af65-6e36ef343882 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.839227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.840226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.840706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.841174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.842070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.845742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.890745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 824/1666] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:45 2022] POST /identity/v3/users => generated 293 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:45.891685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:45.895652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.899864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.900602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.901245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.909903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007e13-ef5e-488b-b789-86b73608a265 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:45.917694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:45.919407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:45.919902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:45.920659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] PATH_INFO: `/v3/projects/6b78673c208546218e5389922cc17f4a/users/2972381ed0074a50bed754c123b0088f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:45.925006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b78673c208546218e5389922cc17f4a, user_id=2972381ed0074a50bed754c123b0088f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.928531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.931983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:45.933848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28007c6-c162-4414-9fad-9bec4143136e service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:45.953478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 825/1667] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:45 2022] PUT /identity/v3/projects/6b78673c208546218e5389922cc17f4a/users/2972381ed0074a50bed754c123b0088f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:45.954808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:46.002954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 843/1668] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:46.020278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.020278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.020278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.155506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] MFA Rules not processed for user `2972381ed0074a50bed754c123b0088f`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:46.164973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:46.167215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dec03-bc81-495a-83e9-ddd12438af9c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:46.178984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:46.180709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.180992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.184084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.186564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:46.188966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:46.227970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 826/1669] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:46.231789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:46.249110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830ba17-4e79-41ea-bb68-304a6cc7d361 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:46.274952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:46.276201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.276585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.277112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.278244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:46.280091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:46.304488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 827/1670] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:46.312131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:46.344488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6c454-812d-4e26-b6a0-f49537eadb50 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:46.346837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 844/1671] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:46.350032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:46.351198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.351477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.351819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] PATH_INFO: `/v3/projects/2ffef5e3170d42518d1a07b083971391/users/b5eb9bfdac1e4135b3abe994443951d8/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.360235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:46.362144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:46.397839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffef5e3170d42518d1a07b083971391, user_id=b5eb9bfdac1e4135b3abe994443951d8, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:46.407085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725878-a19f-4fb5-9e71-249f02797c77 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:46.458740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 828/1672] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:46 2022] PUT /identity/v3/projects/2ffef5e3170d42518d1a07b083971391/users/b5eb9bfdac1e4135b3abe994443951d8/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:46.460617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:46.472609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.473271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.474557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.529676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1b7cf-b656-48e0-bac5-7e1e7110639f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:46.581985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] MFA Rules not processed for user `b5eb9bfdac1e4135b3abe994443951d8`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:46.583396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:46.639252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:46.704313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:46.704962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.705248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.705835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.735760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 829/1673] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:46.744089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:46.844186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2b490-e6f3-420f-a6c6-afe16cf1084d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:46.861661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:46.861661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:46.861661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:46.861661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:46.870553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:46.880970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:46.928930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:46.930429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:46.956948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 845/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:55:46.999573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 830/1675] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:47.003817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.030264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4d006-b9dc-4fab-aa31-a7062be275d1 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.190978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.228523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.230774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.231330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.234972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.241198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:47.244806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:47.316492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 846/1676] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:47 2022] POST /identity/v3/users => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:47.321061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.350470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705859-4a95-4ca6-9e84-f0da7b01f58e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.358497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5457c-e136-4548-8822-a6ccf6b635dd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.364081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.364906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.365465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.365675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.365973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.380471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:47.381826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.382493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.383282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.385582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:47.388982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6ff65-cc46-450b-a887-cb0f705a9174 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:47.423538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 847/1677] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:47.424915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.425205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.425456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.491652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] MFA Rules not processed for user `f2a8b56e48b445709054849f4893767a`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:47.494446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:47.511280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:47.559077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:47.565685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:47.604930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 831/1678] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:47.608560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.644873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e5ae2-f15c-49b1-b708-234ddc41bc22 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.659663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.661263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.662379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.663830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] PATH_INFO: `/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c/users/7968cf8605864db1b4f9c6cc5966921d/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.674041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:47.703500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80a2b563f6d4da8b611b9d6b9bb0a8c, user_id=7968cf8605864db1b4f9c6cc5966921d, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:47.707306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:47.717655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 848/1679] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:47.721172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.723462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:47.744837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 832/1680] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:47 2022] PUT /identity/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c/users/7968cf8605864db1b4f9c6cc5966921d/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:47.746019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:47.750277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:47.758364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd7a38-db7a-4ae0-88b6-be59ddabf948 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.846502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.847237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.847458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.847726 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] PATH_INFO: `/v3/users/4743224ed4eb406aab59e018d69a3116` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.848956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4743224ed4eb406aab59e018d69a3116)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:47.851837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:47.920833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4743224ed4eb406aab59e018d69a3116'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:47.931835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4743224ed4eb406aab59e018d69a3116'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:47.932939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4743224ed4eb406aab59e018d69a3116'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:47.947467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af917208-1200-4448-b63b-52f0e517677e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:47.951953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4743224ed4eb406aab59e018d69a3116'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:47.959760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:47.960659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:47.961265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:47.961553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:47.964274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:47.967143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4743224ed4eb406aab59e018d69a3116'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:47.995230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 849/1681] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:47 2022] DELETE /identity/v3/users/4743224ed4eb406aab59e018d69a3116 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:47.999085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:48.005120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:48.092585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:48.132343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:48.135914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:48.145838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b464a2-b342-4912-9729-6a87ebbdbd51 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:48.152878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:48.154247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.154570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.154906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.155653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:48.157387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc13333-d053-44c0-9e82-7628c0a7e8eb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:48.171183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 850/1682] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:48.173450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.173734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.174554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.175870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 833/1683] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:48.180397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:48.197884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:48.199627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:48.208728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 851/1684] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:48.215345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb215-6006-4a23-b18a-3f287a6f5e76 service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:48.222028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:48.227747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:48.229319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:48.230373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.230658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.230937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] PATH_INFO: `/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c/users/7968cf8605864db1b4f9c6cc5966921d/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.236827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80a2b563f6d4da8b611b9d6b9bb0a8c, user_id=7968cf8605864db1b4f9c6cc5966921d, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:48.242020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5c4a4-e2b1-4040-bfff-1e89a184039c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:48.286944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 834/1685] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:48 2022] PUT /identity/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c/users/7968cf8605864db1b4f9c6cc5966921d/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:48.298103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.298509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.298793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.401405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfcd30-14ab-434a-ab9f-f048985f271f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:48.451756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] MFA Rules not processed for user `7968cf8605864db1b4f9c6cc5966921d`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:48.455570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:48.581537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:48.582054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.582306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.590485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.656484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:48.711406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 835/1686] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:48.721617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:48.724033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:48.801341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:48.803111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:48.821902 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 852/1687] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:48.877392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211b1cc-6f49-4adf-b8ab-caf7dfe5734b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:48.883737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:48.884541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:48.884950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:48.885315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:48.896915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:48.899272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:48.916415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 836/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:49.094736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:49.108220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda81c9-aa49-414a-b2fb-927e8d2efcae service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:49.108323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:49.110162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:49.143111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:49.143523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc806244-35e7-43da-aacb-1e382f8234ad service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:49.148061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:49.148368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:49.148842 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] PATH_INFO: `/v3/users/b5eb9bfdac1e4135b3abe994443951d8` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:49.154153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5eb9bfdac1e4135b3abe994443951d8)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:49.156290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:49.157313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:49.157591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:49.157860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] PATH_INFO: `/v3/projects/79940002af7f4460993cba3e3b4b15de` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:49.163376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79940002af7f4460993cba3e3b4b15de)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:49.165798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:49.165930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:49.225204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:49.270916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79940002af7f4460993cba3e3b4b15de', 'request_id': 'req-a6bc543c-95e8-4369-9597-9190655ee7c2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.287714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79940002af7f4460993cba3e3b4b15de', 'request_id': 'req-a6bc543c-95e8-4369-9597-9190655ee7c2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.300518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79940002af7f4460993cba3e3b4b15de', 'request_id': 'req-a6bc543c-95e8-4369-9597-9190655ee7c2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.301377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5eb9bfdac1e4135b3abe994443951d8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.326189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5eb9bfdac1e4135b3abe994443951d8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.326970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5eb9bfdac1e4135b3abe994443951d8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.344980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] The token cache is being invalidate because project 79940002af7f4460993cba3e3b4b15de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:49.345792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.346525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.349620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 837/1689] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:49 2022] DELETE /identity/v3/projects/79940002af7f4460993cba3e3b4b15de => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:55:49.352532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:49.354365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:49.356027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5eb9bfdac1e4135b3abe994443951d8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.378879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c969da8a-393b-45e3-914d-46a44eee0c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5eb9bfdac1e4135b3abe994443951d8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.387887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 853/1690] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:49 2022] DELETE /identity/v3/users/b5eb9bfdac1e4135b3abe994443951d8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:49.392658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:49.393138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:49.393658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:49.446009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:49.447316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:49.527473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc543c-95e8-4369-9597-9190655ee7c2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:49.579879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:49.605621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 854/1691] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:49.669628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:49.677037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:49.691582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:49.692354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:49.692596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:49.693246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] PATH_INFO: `/v3/users/f7032316aef64d33b00b2b2fa736001f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:49.695688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7032316aef64d33b00b2b2fa736001f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:49.697966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:49.899130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7032316aef64d33b00b2b2fa736001f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.927759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7032316aef64d33b00b2b2fa736001f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.948620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c4ced-a9ce-47f9-b791-ef1641f3eaa3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:49.972084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7032316aef64d33b00b2b2fa736001f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.975027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7032316aef64d33b00b2b2fa736001f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:49.983329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:49.983977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:49.984261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:49.984539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:49.999671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:50.018655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7032316aef64d33b00b2b2fa736001f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.056193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 838/1692] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:49 2022] DELETE /identity/v3/users/f7032316aef64d33b00b2b2fa736001f => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:55:50.056387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:50.057432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e15c2a-b279-4972-9ee4-d79aea311b6a service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:50.087073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 855/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:55:50.341942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:50.344076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:50.509468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3e66c-399e-4fba-8217-41578ac79b5c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:50.521851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:50.522768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:50.523966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:50.524272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] PATH_INFO: `/v3/projects/bd5812ff7859449882b0e82e3b9bb38f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:50.526382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5812ff7859449882b0e82e3b9bb38f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:50.540846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:50.688645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:50.695664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5812ff7859449882b0e82e3b9bb38f', 'request_id': 'req-3175d3a2-9a37-4ca1-9107-db4960b604d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.707048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5812ff7859449882b0e82e3b9bb38f', 'request_id': 'req-3175d3a2-9a37-4ca1-9107-db4960b604d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.734100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5812ff7859449882b0e82e3b9bb38f', 'request_id': 'req-3175d3a2-9a37-4ca1-9107-db4960b604d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.747306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] The token cache is being invalidate because project bd5812ff7859449882b0e82e3b9bb38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:50.747589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.748380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3175d3a2-9a37-4ca1-9107-db4960b604d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:50.755076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 839/1694] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:50 2022] DELETE /identity/v3/projects/bd5812ff7859449882b0e82e3b9bb38f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:50.756952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:50.757575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:50.757575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:50.797782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:50.801476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:50.893806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:50.989073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 856/1695] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:50.990796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:50.991191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:50.991601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.040112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:51.042745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.067701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 840/1696] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:51.073169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.091248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f80ce-812b-4bc3-995a-9d1b86366d6a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.109449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.110870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.111367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.111823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.112716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:51.113234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.120202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:51.158458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 857/1697] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:51.164941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.192679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da0276-2894-48cf-9d6c-da69ca6d1324 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.222163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.222755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.225884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.227104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.227882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.247587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:51.262658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.270949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.308159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f79db-c3e2-439c-ab74-89ba7f690179 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.328721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.331715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.331999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.332277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] PATH_INFO: `/v3/projects/41f16ae5bf064c14b178fb372b9edf2d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.341824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 841/1698] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:51 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:51.342425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f16ae5bf064c14b178fb372b9edf2d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.343558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: WARNING py.warnings [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 00:55:51.343558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: warnings.warn(msg) Aug 30 00:55:51.343558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: Aug 30 00:55:51.344967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.346123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:51.349306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.503117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f16ae5bf064c14b178fb372b9edf2d', 'request_id': 'req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.521549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed766b-5686-43a0-bee4-02cfd1cdecff admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.534134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f16ae5bf064c14b178fb372b9edf2d', 'request_id': 'req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.552337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f16ae5bf064c14b178fb372b9edf2d', 'request_id': 'req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.584658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] The token cache is being invalidate because project 41f16ae5bf064c14b178fb372b9edf2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:51.585243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.587042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.596523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 858/1699] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:51 2022] DELETE /identity/v3/projects/41f16ae5bf064c14b178fb372b9edf2d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:51.596523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.596523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.671687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.672554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.672970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.673387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] PATH_INFO: `/v3/projects/2ffef5e3170d42518d1a07b083971391` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.676538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffef5e3170d42518d1a07b083971391)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.682009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:51.756688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fdb01-1e4e-4bf9-b0b5-06bafdc4efaf admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.775484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffef5e3170d42518d1a07b083971391', 'request_id': 'req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.787940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:51.793898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffef5e3170d42518d1a07b083971391', 'request_id': 'req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.820192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffef5e3170d42518d1a07b083971391', 'request_id': 'req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.851368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] The token cache is being invalidate because project 2ffef5e3170d42518d1a07b083971391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:51.852653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.855497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7a1cbb3c-3cb2-456d-9705-d5c5f1758842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:51.860437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 842/1700] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:51 2022] DELETE /identity/v3/projects/2ffef5e3170d42518d1a07b083971391 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:51.868266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.869592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.870786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.892361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:51.922021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.923176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.924486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.924769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.925429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.927850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:51.930943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.934955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd331c-606f-4fd6-a93b-eca62bc9d864 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:51.959180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 843/1701] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:51.959309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:51.962077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:51.985795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e460de-d139-4f7c-ad2c-269784eac114 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:51.995453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:51.996228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.996441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 859/1702] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:51 2022] POST /identity/v3/users => generated 302 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:51.996592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.997153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] PATH_INFO: `/v3/users/7968cf8605864db1b4f9c6cc5966921d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:51.997876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:51.998167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:51.998357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7968cf8605864db1b4f9c6cc5966921d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:51.998447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.004014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.052999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:52.054946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.067426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 860/1703] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:52.067426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.071790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.098845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7968cf8605864db1b4f9c6cc5966921d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:52.100312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eac246-3c61-4e80-b7ca-0a7f508bdc6e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.106182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.107071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.107676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.109585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.110419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:52.112561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7968cf8605864db1b4f9c6cc5966921d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:52.113476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.127616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7968cf8605864db1b4f9c6cc5966921d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:52.132088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7968cf8605864db1b4f9c6cc5966921d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:52.138383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 861/1704] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:52.139715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.142300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.156317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7968cf8605864db1b4f9c6cc5966921d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:52.176040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34e276-fe12-4b56-989d-0df99e4c2c2a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.181865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 844/1705] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:51 2022] DELETE /identity/v3/users/7968cf8605864db1b4f9c6cc5966921d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:52.185027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.189520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.313183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.314745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.315226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.317130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.318075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:52.320663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:52.322462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.330273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54950cd-34f2-4d90-8351-89d97edfbffc admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.337779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.338666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.339751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.340158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.341222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:52.342986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc18f3-8106-4ca1-bad3-ff093213afa7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.364039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 845/1706] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:52.367773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e68ab-c247-4b1f-b41a-6a5df54d48c8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.368320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e68ab-c247-4b1f-b41a-6a5df54d48c8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.368817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e68ab-c247-4b1f-b41a-6a5df54d48c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.391623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 862/1707] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:52.401079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.417156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3b0e68ab-c247-4b1f-b41a-6a5df54d48c8 None None] MFA Rules not processed for user `e0e5c1345e4049f0bf5fc17ef71b7133`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:52.418615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3b0e68ab-c247-4b1f-b41a-6a5df54d48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.429687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009326e-820a-406a-9c73-1f2b706a8865 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.442958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.443955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.447267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.447267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] PATH_INFO: `/v3/projects/e200111d95cd4740a1b4e38e31fae416/users/3bdfce3f50414e15932dac40cb256486/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.484628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:52.487585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e200111d95cd4740a1b4e38e31fae416, user_id=3bdfce3f50414e15932dac40cb256486, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:52.493401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.532040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 863/1708] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:52 2022] PUT /identity/v3/projects/e200111d95cd4740a1b4e38e31fae416/users/3bdfce3f50414e15932dac40cb256486/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:52.537019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.558399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 846/1709] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:52.560709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb128de-b6c0-4dac-a478-56dbe4aa2ed0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.561180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb128de-b6c0-4dac-a478-56dbe4aa2ed0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.562130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb128de-b6c0-4dac-a478-56dbe4aa2ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.620005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4bb128de-b6c0-4dac-a478-56dbe4aa2ed0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:52.622321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4bb128de-b6c0-4dac-a478-56dbe4aa2ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.695330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2012-4a1c-4a05-80b3-fe40ee3126ef admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.709419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.711299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.711594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.716401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.716401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:52.718354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:52.784233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 847/1710] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:52.786425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d942-59c9-473a-a639-24d6c158965a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.786542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d942-59c9-473a-a639-24d6c158965a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.786794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038d942-59c9-473a-a639-24d6c158965a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.790756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 864/1711] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:52 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:52.795153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:52.850515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8038d942-59c9-473a-a639-24d6c158965a None None] MFA Rules not processed for user `e0e5c1345e4049f0bf5fc17ef71b7133`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:52.852221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8038d942-59c9-473a-a639-24d6c158965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:52.866098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:52.970111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ce9b1-4a4e-4feb-b1cb-6b415479ad4d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:52.995955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:52.996616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:52.996900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:52.997173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:52.997825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.000165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.015448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 848/1712] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:53.015448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d942-59c9-473a-a639-24d6c158965a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.015448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8038d942-59c9-473a-a639-24d6c158965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.064428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 865/1713] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:52 2022] POST /identity/v3/users => generated 284 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:53.065780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.068052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.098296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff9b0a-cba3-43a7-bafc-ae14b935c576 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.116662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.120072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.120553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.122921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] PATH_INFO: `/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.126359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80a2b563f6d4da8b611b9d6b9bb0a8c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.130173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038d942-59c9-473a-a639-24d6c158965a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.130424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.134677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.135351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.136185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.136479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.147090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.148998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.161038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 849/1714] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:53.161916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.163924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.197208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:53.219226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80a2b563f6d4da8b611b9d6b9bb0a8c', 'request_id': 'req-ec0e01a3-1aac-48e4-899a-ad636176df28'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:53.240473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80a2b563f6d4da8b611b9d6b9bb0a8c', 'request_id': 'req-ec0e01a3-1aac-48e4-899a-ad636176df28'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:53.249915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80a2b563f6d4da8b611b9d6b9bb0a8c', 'request_id': 'req-ec0e01a3-1aac-48e4-899a-ad636176df28'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:53.251412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d240f-e669-4736-b9a4-0011d88c4e9b service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.277463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] The token cache is being invalidate because project c80a2b563f6d4da8b611b9d6b9bb0a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:55:53.277912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:53.278790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:53.284810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 866/1715] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:55:53 2022] DELETE /identity/v3/projects/c80a2b563f6d4da8b611b9d6b9bb0a8c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:53.286261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.288825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.342610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.343327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.343615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.343888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.347273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.390578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e01a3-1aac-48e4-899a-ad636176df28 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.396864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.397531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.397965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.398348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.399413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.401309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01e983-6f7d-4888-8ef6-f36cfb8cc2ce admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.416259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 867/1716] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:53.418138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.418587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.419297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.453735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:53.454711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.463412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.465779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58dc6c-5bae-44e1-9d3b-7b7b1c23aad3 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.471121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 868/1717] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:53.473768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.495815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753c13-3ca0-4042-9e1b-1f4643480479 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.496774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 850/1718] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:53.497904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.498196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.498536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.528920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.530120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.530404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.531822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.532501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.540450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.541024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:53.542313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.566300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 851/1719] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:53.571870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.587159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484df80-c359-47da-8ece-9da8367dd0b4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.599928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.601912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.601912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.601912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.602213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:55:53.602864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.604219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.623737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 869/1720] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:53.626950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.646956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 852/1721] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:55:53.648439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.669845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5791fa5-eca4-4f73-8291-d53a81b71bc3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.669973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e0ad-a371-490b-b49b-3443c7c9a28f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:53.691721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.692498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.692784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.693107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] PATH_INFO: `/v3/projects/2e33bcc3e14245a68524c376fb0a6339/users/e7e8c6624e3848f7b49a653d2f1dd032/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.695604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:53.696327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.696615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.696887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.698021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.699906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2c269-b985-4568-8b6c-29dc5ee5c167 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.744901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:53.786723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e33bcc3e14245a68524c376fb0a6339, user_id=e7e8c6624e3848f7b49a653d2f1dd032, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:53.797144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:53.821990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 853/1722] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:53 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:55:53.823646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ef5ca-f092-47be-bf07-512328ad55ca None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:53.823929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ef5ca-f092-47be-bf07-512328ad55ca None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:53.824211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ef5ca-f092-47be-bf07-512328ad55ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:53.854517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 870/1723] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:53 2022] PUT /identity/v3/projects/2e33bcc3e14245a68524c376fb0a6339/users/e7e8c6624e3848f7b49a653d2f1dd032/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:53.856594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:53.967078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6b1ef5ca-f092-47be-bf07-512328ad55ca None None] MFA Rules not processed for user `1d1c13089bd14d48ba637468182eee14`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:53.968533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6b1ef5ca-f092-47be-bf07-512328ad55ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:53.978991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:54.101165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14bcc-af14-4542-86e3-8c036f0b5d57 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.141564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.143969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.144311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.144605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.148748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:54.154264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ac312-200a-4a9a-a288-5ba8163e2ef2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:54.187728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 854/1724] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:54.189404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.190069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.190448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.269256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] MFA Rules not processed for user `e7e8c6624e3848f7b49a653d2f1dd032`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:54.270690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:54.313482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 871/1725] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:53 2022] POST /identity/v3/users => generated 292 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:54.314991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.316155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.320332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.406426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] MFA Rules not processed for user `1d1c13089bd14d48ba637468182eee14`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:54.416443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:54.447918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 872/1726] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:54.449258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.479675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 855/1727] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:54.479675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.479675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:54.482658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654db91-b5a4-4c8e-9a5a-29df4355d8dd None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.491401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.493774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.494424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.494926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.495856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:54.497728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:54.533427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 873/1728] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:54.534908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.552614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da052b6c-5442-4a72-a91e-d30b22de5509 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.558857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.559706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.559991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.560948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.561743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:54.563727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:54.623681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 874/1729] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:54 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:54.624805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.702486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7701a1-1051-4633-85e6-2b63e8fd6525 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.760903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c875-04f7-4d4f-b255-d40da88f3797 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.769258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:54.776377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.782641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.783280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.783732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.784680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:54.789033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:54.830906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 875/1730] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:54.833044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.860644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b20c-ab1b-44b3-bfaf-0122c0756f48 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.875431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.876604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.876604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.876815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.877323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:54.878101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.878864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.879417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547ed17-52d5-44a0-b60b-a814b5b063c1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:54.880383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.880760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.935922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 876/1731] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:54 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:54.937721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0095e1-cbcd-4c82-95d3-a9b70ba909c6 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.938013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0095e1-cbcd-4c82-95d3-a9b70ba909c6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.938285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0095e1-cbcd-4c82-95d3-a9b70ba909c6 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.940649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 877/1732] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 30 00:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:54.941371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0095e1-cbcd-4c82-95d3-a9b70ba909c6 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:54.961612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0095e1-cbcd-4c82-95d3-a9b70ba909c6 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:54.969587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:54.970088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:54.970880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:54.971193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] PATH_INFO: `/v3/projects/c9d7a682bd5b47cc88998c6c76cf32d9/users/f769a1a36f304693bf13e925c9e25c5b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:54.989936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:54.999643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d7a682bd5b47cc88998c6c76cf32d9, user_id=f769a1a36f304693bf13e925c9e25c5b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.002688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.045895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 878/1733] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:54 2022] PUT /identity/v3/projects/c9d7a682bd5b47cc88998c6c76cf32d9/users/f769a1a36f304693bf13e925c9e25c5b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:55.048535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.084093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.086484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1713f-8162-4e6e-9935-23467e4d7907 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.108391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 856/1734] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:55.128035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.137014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.137547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.177740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455e7d-f5cb-4594-90d0-8e0ed64580df admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:55.191375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:55.192024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.192292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.192561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.193222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.195652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e3773-29df-426c-89d1-fe3105fe7b54 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.233302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 879/1735] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:55.238825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd4140-8d6e-4444-9b87-630caa14490b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.239215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd4140-8d6e-4444-9b87-630caa14490b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.239430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd4140-8d6e-4444-9b87-630caa14490b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.281012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:55.284729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:55.320871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-bfbd4140-8d6e-4444-9b87-630caa14490b None None] MFA Rules not processed for user `f769a1a36f304693bf13e925c9e25c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:55.322157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bfbd4140-8d6e-4444-9b87-630caa14490b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:55.473188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 857/1736] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:55.474268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.488711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 880/1737] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:55.490169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.490400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.490672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.504354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6d505-571e-4d8f-952a-eaefd8f935e4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:55.517620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:55.518747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.519420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.521198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] PATH_INFO: `/v3/projects/b7d4afa45e92477eab2c1335b4953ebb/users/47052eac700e4854bf45981520619104/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.565258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] MFA Rules not processed for user `3bdfce3f50414e15932dac40cb256486`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:55.566513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:55.568981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d4afa45e92477eab2c1335b4953ebb, user_id=47052eac700e4854bf45981520619104, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.577021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.614904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 858/1738] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:55 2022] PUT /identity/v3/projects/b7d4afa45e92477eab2c1335b4953ebb/users/47052eac700e4854bf45981520619104/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:55.614904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.731962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 881/1739] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:55.731962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.733392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:55.740296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:55.758111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbfcc7-6d7a-49f5-b3f4-3c8e726c32a3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:55.769081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:55.783236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e6a6-09dd-41c1-87df-bb0cb9a801e1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:55.800864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:55.801638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.802114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.802495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.803306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.807991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.886671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:55.887255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 859/1740] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:55 2022] POST /identity/v3/projects => generated 381 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:55.887355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:55.887355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:55.887500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.887662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:55.903880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:55.905361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:55.929832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 882/1741] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:55.930983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:55.932733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:55.970002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c984e-b43f-4f27-8682-d73488b3b21b service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.046990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae683c-0eda-4bf3-ac4a-13b9d0a33e1c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.060494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.061150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.061436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.063286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.064022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.065750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.083285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 860/1742] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:56.084289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:56.099381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.099939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.100222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.100688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.101270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.109052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c39f26-4313-4ceb-abb1-a01a03d3711d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.117318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a20f7-c2ac-4436-bbfe-c2937f59a254 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.130028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.132371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.132678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.133114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.133788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.140437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.181548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 883/1743] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:55:55 2022] POST /identity/v3/projects => generated 405 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:56.181548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da2d9-b331-40f6-8cca-049baafd2563 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.181548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da2d9-b331-40f6-8cca-049baafd2563 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.181903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da2d9-b331-40f6-8cca-049baafd2563 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.185531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 884/1744] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 30 00:55:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:55:56.195496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da2d9-b331-40f6-8cca-049baafd2563 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:56.218430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 861/1745] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:56.218430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:56.281696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7be8d3-0051-4475-a096-c466724d9065 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.418519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da2d9-b331-40f6-8cca-049baafd2563 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.446895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.447633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.447913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.448189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] PATH_INFO: `/v3/projects/b7d4afa45e92477eab2c1335b4953ebb/users/47052eac700e4854bf45981520619104/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.454370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d4afa45e92477eab2c1335b4953ebb, user_id=47052eac700e4854bf45981520619104, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.457145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f2d9-6ea8-468f-91c0-181cd21388f9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.492598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.493279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.493559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.493830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.495633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.500143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 885/1746] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:56 2022] PUT /identity/v3/projects/b7d4afa45e92477eab2c1335b4953ebb/users/47052eac700e4854bf45981520619104/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:56.505841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.506321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.506876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.510559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc27d5-00f7-4b3d-ae29-4af7be147b2d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.562789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:56.565470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 862/1747] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/users => generated 309 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:55:56.566075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:56.568750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474b22-2a5e-459a-bb16-bb959eb38255 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.570103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474b22-2a5e-459a-bb16-bb959eb38255 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.570392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474b22-2a5e-459a-bb16-bb959eb38255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.625402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b3474b22-2a5e-459a-bb16-bb959eb38255 None None] MFA Rules not processed for user `f2a8b56e48b445709054849f4893767a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:56.629673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b3474b22-2a5e-459a-bb16-bb959eb38255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:56.688267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:56.704313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:56.749359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 886/1748] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:56.750541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:56.773461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 863/1749] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:56.774860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.775107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.779632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.827901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] MFA Rules not processed for user `22f11e29c4ad4712a54877b0895d28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:56.829049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:56.902188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c02019-2bca-4695-81f7-c43e16293c14 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:56.914184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:56.914843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.915149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.915426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.920230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:56.923268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d81229-1f4b-4476-ab4e-c06d723f3061 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:56.947128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 887/1750] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:56.954060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:56.954357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:56.954620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:56.991362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 864/1751] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:56.994806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.014467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1ccb-d28a-4401-ae66-1c6318870a90 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.025508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.026469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.027606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] MFA Rules not processed for user `47052eac700e4854bf45981520619104`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:57.028570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.029260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.030070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:57.031673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.032883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.063103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 865/1752] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:57.064249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.110896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17d1d-57e0-4893-9502-b4a53899e6bd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.131949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.135367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.139336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.139525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.155633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.157982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.169110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 866/1753] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:57.170255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.188846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4079a0-632d-46e0-9f01-242659b8ea7d service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.195266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.196413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.196835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.197167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] PATH_INFO: `/v3/projects/70923533ccf743b081e7586812b5173b/users/aec96921bf3547808aff3178fd3ddf9b/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.216557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70923533ccf743b081e7586812b5173b, user_id=aec96921bf3547808aff3178fd3ddf9b, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.219488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.237806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 867/1754] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:57 2022] PUT /identity/v3/projects/70923533ccf743b081e7586812b5173b/users/aec96921bf3547808aff3178fd3ddf9b/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:57.239115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.263579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 888/1755] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:55:57.265050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.271386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:57.338789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6c370-50d4-4151-af12-fddb92c98284 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.345815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.346798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.347853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.348374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] PATH_INFO: `/v3/projects/8fc54bc4b0d64b89a762a062871e0163/users/6dca7d838def48b1aa84bb5a2e86e5ac/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.368809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc54bc4b0d64b89a762a062871e0163, user_id=6dca7d838def48b1aa84bb5a2e86e5ac, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.373215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.399594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 868/1756] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:57 2022] PUT /identity/v3/projects/8fc54bc4b0d64b89a762a062871e0163/users/6dca7d838def48b1aa84bb5a2e86e5ac/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:57.401246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.410593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:57.427799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:57.441751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8ff19-b55b-4618-acd1-12f34074c105 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.547572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde0c83-8103-410d-be15-81385c842d91 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.553645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.554612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.554612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.554895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.556690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.559450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.577015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 869/1757] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:57.577015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.577015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:57.622108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.622981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.623395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.623718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.626841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:57.653240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21279f08-a8d3-42a4-bea5-58c61f1da23f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.657306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.658605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.659269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.659549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.659990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.660588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.688881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 889/1758] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:57.704567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.739684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10509f-4a9d-4a99-8bb6-21c11d3fbed8 service placement] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.745050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.746961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.774545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.777161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 870/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:57.778882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.779504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.779973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.781402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.782859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.790804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.822787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc281c-29cd-4d61-96d3-4c725c536907 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.830452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 890/1760] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:55:57.839725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:57.854750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.857209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.859773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.860785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] PATH_INFO: `/v3/projects/70923533ccf743b081e7586812b5173b/users/aec96921bf3547808aff3178fd3ddf9b/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.871122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70923533ccf743b081e7586812b5173b, user_id=aec96921bf3547808aff3178fd3ddf9b, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.878738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13793ca-6b3b-42e4-9178-4d39fcfbe173 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.886344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1cf7d-9afe-4d9a-a707-ef97dd5f8679 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:57.897099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:57.900494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.900916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.901778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] PATH_INFO: `/v3/projects/8fc54bc4b0d64b89a762a062871e0163/users/6dca7d838def48b1aa84bb5a2e86e5ac/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.915971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc54bc4b0d64b89a762a062871e0163, user_id=6dca7d838def48b1aa84bb5a2e86e5ac, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:57.916905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 871/1761] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:57 2022] PUT /identity/v3/projects/70923533ccf743b081e7586812b5173b/users/aec96921bf3547808aff3178fd3ddf9b/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:57.917003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:57.917950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8275-2664-44f8-80cd-75852149e337 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:57.918365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8275-2664-44f8-80cd-75852149e337 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:57.918726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f8275-2664-44f8-80cd-75852149e337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:57.942718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 891/1762] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:55:57 2022] PUT /identity/v3/projects/8fc54bc4b0d64b89a762a062871e0163/users/6dca7d838def48b1aa84bb5a2e86e5ac/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:55:57.964568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-186f8275-2664-44f8-80cd-75852149e337 None None] MFA Rules not processed for user `aec96921bf3547808aff3178fd3ddf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:55:57.966016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-186f8275-2664-44f8-80cd-75852149e337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:57.995422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:58.044274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:58.185148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4f0e-e44d-4479-9cd9-f915b80a98d0 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:58.194925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:58.197277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:58.198645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:58.199785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:58.207953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 872/1763] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:55:58.246208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f8275-2664-44f8-80cd-75852149e337 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:58.248650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-186f8275-2664-44f8-80cd-75852149e337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:58.287491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f8275-2664-44f8-80cd-75852149e337 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:58.306406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:58.310984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:58.311374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:58.311722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:58.368497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:58.371701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:58.425257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 873/1764] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:58.455579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:58.455579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:58.487498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 892/1765] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:55:58.742983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:59.151270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd969d2-2b92-44c4-bdfe-3c56a0358682 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:59.201587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:59.205146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:59.208852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:59.209272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:59.456646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:59.460896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:55:59.484055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:59.485714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:59.538297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:59.552481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 874/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:59.554934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:55:59.618064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb2ba-e555-4d85-928e-82d78129a49c service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:59.643519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:59.644238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:59.644524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:59.644805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:59.667896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947c17e-d3c9-4b54-bd18-4b03b5a87c39 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:55:59.680396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:55:59.683808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:55:59.684646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:55:59.684980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] PATH_INFO: `/v3/users/c1421170837d4168878252e3b2a2fae6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:55:59.689183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1421170837d4168878252e3b2a2fae6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:59.700047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:59.880119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:55:59.901622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1421170837d4168878252e3b2a2fae6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:59.950490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1421170837d4168878252e3b2a2fae6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:59.954150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1421170837d4168878252e3b2a2fae6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:55:59.955274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:55:59.958519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dd3a6-701a-4c7d-88d3-b0d9ae923e85 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:55:59.986581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 875/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:55:59.992912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1421170837d4168878252e3b2a2fae6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:00.017593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5c61abc6-280a-4896-a767-099532211855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1421170837d4168878252e3b2a2fae6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:00.034123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 893/1768] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:55:59 2022] DELETE /identity/v3/users/c1421170837d4168878252e3b2a2fae6 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:00.041648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:00.041941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:00.043788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:00.091179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:00.095978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:00.108930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3ee-d67d-473a-9d97-55240c413806 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:00.109223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3ee-d67d-473a-9d97-55240c413806 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:00.109503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3ee-d67d-473a-9d97-55240c413806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:00.176250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-439ee3ee-d67d-473a-9d97-55240c413806 None None] MFA Rules not processed for user `1d1c13089bd14d48ba637468182eee14`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:00.180488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-439ee3ee-d67d-473a-9d97-55240c413806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:00.450676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 876/1769] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:00.489543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:00.494776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:00.544935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 894/1770] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:00.696004 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:00.696343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:00.696673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:00.762725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56664203-cce5-491f-9ad9-01e54a964fa2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:00.784499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:00.785767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:00.785767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:00.785767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:00.825625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:00.827124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:00.836811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] MFA Rules not processed for user `6dca7d838def48b1aa84bb5a2e86e5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:00.838095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:00.873991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 877/1771] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:00.919742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:01.006187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 895/1772] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:02.019734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:02.022684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:02.057821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06067d76-d698-4357-81a8-461be5c76b1f service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:02.072832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:02.073499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:02.073786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:02.075531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] PATH_INFO: `/v3/projects/95a75c155ecb4895a57e8824a4a7f1e3` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:02.077221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a75c155ecb4895a57e8824a4a7f1e3)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:02.079940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:02.259708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a75c155ecb4895a57e8824a4a7f1e3', 'request_id': 'req-5081d1b9-2c70-476a-a5a9-d31b2645448b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.276966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a75c155ecb4895a57e8824a4a7f1e3', 'request_id': 'req-5081d1b9-2c70-476a-a5a9-d31b2645448b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.305301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a75c155ecb4895a57e8824a4a7f1e3', 'request_id': 'req-5081d1b9-2c70-476a-a5a9-d31b2645448b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.323947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] The token cache is being invalidate because project 95a75c155ecb4895a57e8824a4a7f1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:02.324429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.352278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5081d1b9-2c70-476a-a5a9-d31b2645448b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.361298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 878/1773] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:02 2022] DELETE /identity/v3/projects/95a75c155ecb4895a57e8824a4a7f1e3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:02.362753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:02.382024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6c040-5ebe-4d9d-a024-21a30f284763 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:02.504991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:02.506196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:02.506774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:02.507648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] PATH_INFO: `/v3/users/2972381ed0074a50bed754c123b0088f` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:02.510903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2972381ed0074a50bed754c123b0088f)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:02.514269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:02.634096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2972381ed0074a50bed754c123b0088f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.659900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2972381ed0074a50bed754c123b0088f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.660974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2972381ed0074a50bed754c123b0088f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.767251 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2972381ed0074a50bed754c123b0088f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.792774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2972381ed0074a50bed754c123b0088f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:02.803736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 896/1774] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:02 2022] DELETE /identity/v3/users/2972381ed0074a50bed754c123b0088f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:03.212140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.213012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.213868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.261136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:03.360046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:03.361674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.553059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:03.556776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:03.568044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 879/1775] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:03.719160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:03.751649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f302723-24e8-4aff-a6ce-b67acf2f2d61 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:03.758833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:03.760298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:03.760853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:03.761138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] PATH_INFO: `/v3/users/f769a1a36f304693bf13e925c9e25c5b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:03.762391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f769a1a36f304693bf13e925c9e25c5b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:03.798194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:03.953783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:03.955825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.010151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f769a1a36f304693bf13e925c9e25c5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:04.029287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f769a1a36f304693bf13e925c9e25c5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:04.030402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f769a1a36f304693bf13e925c9e25c5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:04.055045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f769a1a36f304693bf13e925c9e25c5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:04.075616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-33187959-e623-409d-9dad-1b6f1fb1e9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f769a1a36f304693bf13e925c9e25c5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:04.086328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 897/1776] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:03 2022] DELETE /identity/v3/users/f769a1a36f304693bf13e925c9e25c5b => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:04.088123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d994e0-3ed0-4ab3-af95-d2550f5930a4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.088569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d994e0-3ed0-4ab3-af95-d2550f5930a4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.089157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d994e0-3ed0-4ab3-af95-d2550f5930a4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.091068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 898/1777] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:04.092295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5397-772f-4c53-bf4c-35bf9047dd53 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.092712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5397-772f-4c53-bf4c-35bf9047dd53 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.100540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5397-772f-4c53-bf4c-35bf9047dd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.127543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c47f5397-772f-4c53-bf4c-35bf9047dd53 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:04.128957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c47f5397-772f-4c53-bf4c-35bf9047dd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.201545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a778f-7ff7-468d-8201-8dc8ee36f3dd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:04.248674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 899/1778] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:04.250461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.250875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.251289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.291494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:04.293191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.321877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:04.337426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.338073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.343216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.347724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.383579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:04.447685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 900/1779] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:04.448562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:04.449896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.465540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:04.576473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:04.578822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:04.595514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc82ca-fca3-4482-a0bf-d677e6b05885 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:04.603753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:04.604676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.605148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.605567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.629277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:04.629934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 880/1780] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:56:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:56:04.631190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:04.631428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:04.636260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:04.646789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 901/1781] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:04.782339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f2827-32cc-4a45-8519-cef10be015de service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:04.815972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:04.816791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:04.817082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:04.823715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:04.873041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:04.885975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:04.931340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 881/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:06.288019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:06.290253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.316648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f7e2-9a8e-4e57-b74f-3f036192e848 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:06.330555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:06.332236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:06.332724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:06.333888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] PATH_INFO: `/v3/users/e7e8c6624e3848f7b49a653d2f1dd032` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.339058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e8c6624e3848f7b49a653d2f1dd032)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:06.341386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:06.482944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e8c6624e3848f7b49a653d2f1dd032'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.507800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e8c6624e3848f7b49a653d2f1dd032'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.509458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e8c6624e3848f7b49a653d2f1dd032'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.534819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:06.536799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.550152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e8c6624e3848f7b49a653d2f1dd032'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.571668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a11ae12b-5b66-40aa-9fc1-9457d6bfee56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e8c6624e3848f7b49a653d2f1dd032'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.583917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 902/1783] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:06 2022] DELETE /identity/v3/users/e7e8c6624e3848f7b49a653d2f1dd032 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:06.585741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f489c-04c7-4ebe-a92e-89325246a58c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:06.587065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435337-a4f3-475e-8822-0ba8e6ac72a2 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:06.587661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f489c-04c7-4ebe-a92e-89325246a58c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:06.588108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f489c-04c7-4ebe-a92e-89325246a58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.641642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9b2f489c-04c7-4ebe-a92e-89325246a58c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:06.645322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9b2f489c-04c7-4ebe-a92e-89325246a58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.720031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:06.720735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:06.721171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:06.721566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] PATH_INFO: `/v3/projects/6b78673c208546218e5389922cc17f4a` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.723991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b78673c208546218e5389922cc17f4a)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:06.730271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:06.867442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b78673c208546218e5389922cc17f4a', 'request_id': 'req-f6eb18e1-83b3-4607-8d17-db1b6193d73b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.895139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b78673c208546218e5389922cc17f4a', 'request_id': 'req-f6eb18e1-83b3-4607-8d17-db1b6193d73b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.915457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b78673c208546218e5389922cc17f4a', 'request_id': 'req-f6eb18e1-83b3-4607-8d17-db1b6193d73b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.926353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 903/1784] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:06.930988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d51a-60b0-41ca-b100-1d707224b8cd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:06.932699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d51a-60b0-41ca-b100-1d707224b8cd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:06.933263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d51a-60b0-41ca-b100-1d707224b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:06.964033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] The token cache is being invalidate because project 6b78673c208546218e5389922cc17f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:06.966314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.967341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:06.974267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 882/1785] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:06 2022] DELETE /identity/v3/projects/6b78673c208546218e5389922cc17f4a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:06.975289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:06.979712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:06.997514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-39e0d51a-60b0-41ca-b100-1d707224b8cd None None] MFA Rules not processed for user `22f11e29c4ad4712a54877b0895d28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:07.001532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-39e0d51a-60b0-41ca-b100-1d707224b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:07.053334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:07.056337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:07.171159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb18e1-83b3-4607-8d17-db1b6193d73b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:07.175352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:07.175988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.176266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.176540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.179309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:07.180792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 904/1786] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:07.182871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.183528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.183876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.222104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:07.227956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:07.320667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:07.322789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e9071-409e-47e3-b182-8c9312bbe93a service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:07.337230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 883/1787] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:07.445265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 905/1788] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:07.454673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.454990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.455318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.507401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:07.512350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:07.542684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 884/1789] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:07.544957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:07.562888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1927e2-3c24-4354-b2f6-15c8ec1da896 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:07.619671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:07.621216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.622478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.623561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.627428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:07.628408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:07.632785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:07.707544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:07.715982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 906/1790] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:07.725444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139804-6b49-45d1-b3fc-983fecd83506 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:07.745882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:07.746525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.746820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.747155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.747872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:07.755290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:07.823719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 885/1791] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:07 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:07.830458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:07.863624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41631b5a-2ad2-4135-9089-916da617cc32 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:07.978630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:07.979573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:07.980008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:07.980579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:07.981429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:07.983852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:08.020738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 907/1792] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:07 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:08.021075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:08.046804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c75f0d-760d-469a-bd2f-c61a46b08204 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:08.056994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:08.057557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.057842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.058124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.059293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:08.064722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:08.099025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:08.100520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 886/1793] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:08.115385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13941c83-3bef-4780-b9f5-3d6852272d86 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:08.129024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:08.129858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.130238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.130626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.131965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:08.136507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd50e-cb8f-43b0-8336-f8db2dbccd9d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:08.164810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 908/1794] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:08.171795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:08.198991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8a335-1424-4e51-9680-6f4e9b408af6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:08.212148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:08.213163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.213471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.213738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] PATH_INFO: `/v3/projects/0f39010f618c467993f15da835cb51ed/users/6b42086b77a441159f439108cebd8dec/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.253076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f39010f618c467993f15da835cb51ed, user_id=6b42086b77a441159f439108cebd8dec, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:08.258352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:08.318235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 887/1795] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:08 2022] PUT /identity/v3/projects/0f39010f618c467993f15da835cb51ed/users/6b42086b77a441159f439108cebd8dec/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:08.334305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f595-29d5-40b9-aeb1-e63ba873567b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.335779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f595-29d5-40b9-aeb1-e63ba873567b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.336333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f595-29d5-40b9-aeb1-e63ba873567b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.400835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:08.403122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:08.435585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ea20f595-29d5-40b9-aeb1-e63ba873567b None None] MFA Rules not processed for user `6b42086b77a441159f439108cebd8dec`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:08.436258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ea20f595-29d5-40b9-aeb1-e63ba873567b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:08.497823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:08.656099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afef6d-eca0-40e0-a428-8a804772d680 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:08.692867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:08.693639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.696330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 909/1796] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:08.698111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f425f-ac24-4730-9615-3774a495a83a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:08.698456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f425f-ac24-4730-9615-3774a495a83a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.698746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f425f-ac24-4730-9615-3774a495a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.703652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:08.703969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] PATH_INFO: `/v3/projects/c9d7a682bd5b47cc88998c6c76cf32d9` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:08.713657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d7a682bd5b47cc88998c6c76cf32d9)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:08.716003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:08.752758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:08.792721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-997f425f-ac24-4730-9615-3774a495a83a None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:08.798598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-997f425f-ac24-4730-9615-3774a495a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:08.861543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d7a682bd5b47cc88998c6c76cf32d9', 'request_id': 'req-d456d72b-c51e-4a25-a794-e50d2376584d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:08.899211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d7a682bd5b47cc88998c6c76cf32d9', 'request_id': 'req-d456d72b-c51e-4a25-a794-e50d2376584d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:08.918798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d7a682bd5b47cc88998c6c76cf32d9', 'request_id': 'req-d456d72b-c51e-4a25-a794-e50d2376584d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:08.941687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] The token cache is being invalidate because project c9d7a682bd5b47cc88998c6c76cf32d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:08.943287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:08.945623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:08.952493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 888/1797] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:08 2022] DELETE /identity/v3/projects/c9d7a682bd5b47cc88998c6c76cf32d9 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:08.953279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:08.956614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.017175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 910/1798] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Aug 30 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:09.020294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f425f-ac24-4730-9615-3774a495a83a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.023108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-997f425f-ac24-4730-9615-3774a495a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.052615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f425f-ac24-4730-9615-3774a495a83a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.176798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d72b-c51e-4a25-a794-e50d2376584d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.190052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.191749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.192758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.194374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.197294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.200800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.201692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.202518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.202891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.204375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] PATH_INFO: `/v3/users/3bdfce3f50414e15932dac40cb256486` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.208068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdfce3f50414e15932dac40cb256486)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.211917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.265491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 889/1799] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:08 2022] POST /identity/v3/projects => generated 391 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:09.267962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.271991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.361886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdfce3f50414e15932dac40cb256486'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.378643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdfce3f50414e15932dac40cb256486'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.379486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdfce3f50414e15932dac40cb256486'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.407650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdfce3f50414e15932dac40cb256486'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.412646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:09.422368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdfce3f50414e15932dac40cb256486'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.433472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 911/1800] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:09 2022] DELETE /identity/v3/users/3bdfce3f50414e15932dac40cb256486 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:09.434275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.435739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.543876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d72c6-fb6b-4783-aff4-c0c8973c52e2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.599693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d3875-10b3-4bc7-b458-c897fe9a9d25 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.611321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.612962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.613291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.613648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.619845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.622288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:09.653464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.656600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.671366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 912/1801] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:09.672639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.680656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.722270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.722929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.723251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.723544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] PATH_INFO: `/v3/users/aec96921bf3547808aff3178fd3ddf9b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.727790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec96921bf3547808aff3178fd3ddf9b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.738180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.802704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fc652-2808-4b1b-a8c7-c543a050bd54 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:09.813359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:09.815852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:09.816496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:09.817489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:09.823399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:09.828290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:09.833857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec96921bf3547808aff3178fd3ddf9b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.844513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec96921bf3547808aff3178fd3ddf9b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.856962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec96921bf3547808aff3178fd3ddf9b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.859056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec96921bf3547808aff3178fd3ddf9b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.900599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec96921bf3547808aff3178fd3ddf9b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:09.911270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 890/1802] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:09 2022] DELETE /identity/v3/users/aec96921bf3547808aff3178fd3ddf9b => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:09.911270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.916455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:09.973872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 913/1803] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:09 2022] POST /identity/v3/users => generated 302 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:09.975820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:09.983778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:10.014266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416faf51-cc31-4fda-8759-bdc5d8f88a7e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:10.225765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ead8a-6d67-4ed3-b276-8ba3de5f5cbf admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:10.258541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:10.261365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:10.265634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:10.270289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:10.281974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:10.318552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:10.322408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:10.324347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:10.325585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:10.326124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:10.332827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:10.355525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:10.359070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:10.365958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 891/1804] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:10.370289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:10.373213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:10.389353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 914/1805] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:10.396738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:10.402685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:10.480274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:10.553078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:10.573193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ca4b8-0f12-4f44-8c40-813e33d1eaf0 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:10.578235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469cf13-1d29-489f-8f6a-631e688b0e7b service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:10.584705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:10.586161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:10.586503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:10.586743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:10.602285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:10.603122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:10.605250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:10.608733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:10.609101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] PATH_INFO: `/v3/projects/2e33bcc3e14245a68524c376fb0a6339` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:10.611088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:10.612339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e33bcc3e14245a68524c376fb0a6339)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:10.631295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:10.669602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 892/1806] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:10.674379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:10.706650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d06e3c-32e0-4a63-8703-62cec6a3c3c0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:10.718720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:10.720906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:10.721350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:10.722016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] PATH_INFO: `/v3/projects/df726c0a470144ea8b3490aadef8dc7b/users/58c0f874cbd5464ea9565a810a989aae/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:10.786150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df726c0a470144ea8b3490aadef8dc7b, user_id=58c0f874cbd5464ea9565a810a989aae, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:10.790254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:10.798947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e33bcc3e14245a68524c376fb0a6339', 'request_id': 'req-38e00307-2a86-480d-92cc-8e84785b5aa3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:10.813327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e33bcc3e14245a68524c376fb0a6339', 'request_id': 'req-38e00307-2a86-480d-92cc-8e84785b5aa3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:10.826421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e33bcc3e14245a68524c376fb0a6339', 'request_id': 'req-38e00307-2a86-480d-92cc-8e84785b5aa3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:10.854467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] The token cache is being invalidate because project 2e33bcc3e14245a68524c376fb0a6339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:10.854779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:10.855558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:10.857343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 893/1807] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:10 2022] PUT /identity/v3/projects/df726c0a470144ea8b3490aadef8dc7b/users/58c0f874cbd5464ea9565a810a989aae/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:10.857491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 915/1808] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:10 2022] DELETE /identity/v3/projects/2e33bcc3e14245a68524c376fb0a6339 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:10.861799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:10.861987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:10.863844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:10.865397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:11.039061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d90f6-8e76-4e5c-9cee-e4b6e96cd229 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:11.051367 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:11.052279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.052719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.053018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] PATH_INFO: `/v3/users/47052eac700e4854bf45981520619104` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.070168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47052eac700e4854bf45981520619104)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:11.074688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:11.096507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e00307-2a86-480d-92cc-8e84785b5aa3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:11.115276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:11.123078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.123078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.123078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.126563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:11.133701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:11.176760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 916/1809] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:11.182413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:11.206734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47052eac700e4854bf45981520619104'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:11.210544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ee69b-50da-4108-936a-dc7d22778c42 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:11.225635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47052eac700e4854bf45981520619104'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:11.245886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47052eac700e4854bf45981520619104'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:11.246567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47052eac700e4854bf45981520619104'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:11.247790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:11.253683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.254969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.259486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] PATH_INFO: `/v3/projects/df726c0a470144ea8b3490aadef8dc7b/users/58c0f874cbd5464ea9565a810a989aae/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.272502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47052eac700e4854bf45981520619104'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:11.272744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df726c0a470144ea8b3490aadef8dc7b, user_id=58c0f874cbd5464ea9565a810a989aae, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:11.281011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175a3c0-4211-4f73-822a-73df32ae4a03 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:11.310109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 917/1810] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:11 2022] PUT /identity/v3/projects/df726c0a470144ea8b3490aadef8dc7b/users/58c0f874cbd5464ea9565a810a989aae/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:11.315954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 894/1811] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:10 2022] DELETE /identity/v3/users/47052eac700e4854bf45981520619104 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:11.328728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.329012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.329292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.412674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] MFA Rules not processed for user `58c0f874cbd5464ea9565a810a989aae`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:11.414146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:11.473893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:11.640609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 918/1812] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:11.661751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:11.669293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:11.755434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:11.772295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432725a2-79f9-4b94-b35d-c34dac266d94 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:11.778150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:11.778850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:11.779178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:11.779435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:11.798453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:11.800604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:11.887110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 895/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:12.050562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:12.053925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:12.085178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a09b8-e837-412e-9326-abadafbe8bfe None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:12.107896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:12.109233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:12.109981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:12.110536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:12.114374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:12.226725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:12.228001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:12.228460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:12.230639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:12.246332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 919/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:12.371460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6905dc-fb89-448d-ab43-78a8e48f0210 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:12.382285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:12.384598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:12.385010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:12.385317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] PATH_INFO: `/v3/projects/e200111d95cd4740a1b4e38e31fae416` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:12.401510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e200111d95cd4740a1b4e38e31fae416)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:12.407666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:12.540434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e200111d95cd4740a1b4e38e31fae416', 'request_id': 'req-cc9015b4-bafe-44d0-828e-d7b899df72ce'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:12.549313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e200111d95cd4740a1b4e38e31fae416', 'request_id': 'req-cc9015b4-bafe-44d0-828e-d7b899df72ce'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:12.564357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e200111d95cd4740a1b4e38e31fae416', 'request_id': 'req-cc9015b4-bafe-44d0-828e-d7b899df72ce'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:12.600300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] The token cache is being invalidate because project e200111d95cd4740a1b4e38e31fae416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:12.601737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:12.602925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc9015b4-bafe-44d0-828e-d7b899df72ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:12.609195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 896/1815] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:12 2022] DELETE /identity/v3/projects/e200111d95cd4740a1b4e38e31fae416 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:12.610354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:12.642844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2582b63-3b4b-46a9-9c12-ebd8c1dcceeb service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:12.838994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:12.839740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:12.840021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:12.840293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] PATH_INFO: `/v3/users/6dca7d838def48b1aa84bb5a2e86e5ac` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:12.844923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dca7d838def48b1aa84bb5a2e86e5ac)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:12.846959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:13.010434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dca7d838def48b1aa84bb5a2e86e5ac'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:13.034494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dca7d838def48b1aa84bb5a2e86e5ac'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:13.035886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dca7d838def48b1aa84bb5a2e86e5ac'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:13.068154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:13.076541 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dca7d838def48b1aa84bb5a2e86e5ac'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:13.102973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dca7d838def48b1aa84bb5a2e86e5ac'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:13.112509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 920/1816] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:12 2022] DELETE /identity/v3/users/6dca7d838def48b1aa84bb5a2e86e5ac => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:13.957297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7174347-b7ae-422e-b2a5-218678964c9b None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:13.963308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7174347-b7ae-422e-b2a5-218678964c9b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:13.963308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7174347-b7ae-422e-b2a5-218678964c9b None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:13.963308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 897/1817] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:13.987137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:13.988650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.109708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.109990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.110280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.159680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:14.160893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.191929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2df99c-c178-4d4b-be5c-afe953a0342f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:14.210915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:14.211625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.211974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.212158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] PATH_INFO: `/v3/projects/70923533ccf743b081e7586812b5173b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.215184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=70923533ccf743b081e7586812b5173b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:14.238588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:14.347614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 898/1818] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:14.348633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:14.350197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.379917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd482d-59e3-4371-bb03-cce16558ffc4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:14.403476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70923533ccf743b081e7586812b5173b', 'request_id': 'req-47139b4a-98f1-4259-84d9-98e5aafb39dc'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.407222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:14.440181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70923533ccf743b081e7586812b5173b', 'request_id': 'req-47139b4a-98f1-4259-84d9-98e5aafb39dc'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.458973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70923533ccf743b081e7586812b5173b', 'request_id': 'req-47139b4a-98f1-4259-84d9-98e5aafb39dc'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.542699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:14.543548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.543845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.544135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] PATH_INFO: `/v3/projects/8fc54bc4b0d64b89a762a062871e0163` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.545566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc54bc4b0d64b89a762a062871e0163)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:14.546342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] The token cache is being invalidate because project 70923533ccf743b081e7586812b5173b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:14.547775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.550837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47139b4a-98f1-4259-84d9-98e5aafb39dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.556262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 921/1819] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:13 2022] DELETE /identity/v3/projects/70923533ccf743b081e7586812b5173b => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:14.558479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1494b-84af-4f60-ad25-0857314ddbe8 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.559820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1494b-84af-4f60-ad25-0857314ddbe8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.560436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1494b-84af-4f60-ad25-0857314ddbe8 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.562980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:14.567250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 922/1820] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:56:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:14.569813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5795-9b96-47f6-8530-35937e37c9de None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.570307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5795-9b96-47f6-8530-35937e37c9de None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.570966 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5795-9b96-47f6-8530-35937e37c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.602354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f99a5795-9b96-47f6-8530-35937e37c9de None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:14.604361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f99a5795-9b96-47f6-8530-35937e37c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.615985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 923/1821] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:14.623802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60513e7-8ba6-43f3-9cd9-abadbc77c3af None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.624424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60513e7-8ba6-43f3-9cd9-abadbc77c3af None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.624844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60513e7-8ba6-43f3-9cd9-abadbc77c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.677154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc54bc4b0d64b89a762a062871e0163', 'request_id': 'req-d76e2a86-7969-4721-8777-8ff5facac7fb'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.695864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc54bc4b0d64b89a762a062871e0163', 'request_id': 'req-d76e2a86-7969-4721-8777-8ff5facac7fb'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.721305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc54bc4b0d64b89a762a062871e0163', 'request_id': 'req-d76e2a86-7969-4721-8777-8ff5facac7fb'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.734868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d60513e7-8ba6-43f3-9cd9-abadbc77c3af None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:14.736066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d60513e7-8ba6-43f3-9cd9-abadbc77c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.744780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] The token cache is being invalidate because project 8fc54bc4b0d64b89a762a062871e0163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:14.746027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.749258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d76e2a86-7969-4721-8777-8ff5facac7fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:14.751341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 899/1822] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:14 2022] DELETE /identity/v3/projects/8fc54bc4b0d64b89a762a062871e0163 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:14.753243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.753766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.754334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.807617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:14.810587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:14.919553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 924/1823] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:14.924140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:14.924432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:14.924710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:14.963646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:15.002377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:15.008422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.017875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 900/1824] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:15.018778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.031239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 925/1825] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:15.032090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.034326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.040915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0296d7-7d63-4f89-aba2-f512d1f2eef6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.046405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.047175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.047522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.048035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.049814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:15.050235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.052219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933134-82fc-4048-977b-fc00a8cd63cf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.060712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bbc7c-0964-423f-b1af-d9ba2c6503fd None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.074799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 901/1826] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:15.075030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.075030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.075030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.075030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.075333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.075333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.075463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.075536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.091531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.093662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.106303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:15.107959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.109028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 926/1827] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:15.122419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.143603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 902/1828] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:15.145785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.146744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a8321-df89-4c77-8d30-1894849e9833 service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.159209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.159947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.160246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.160608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.161339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.161911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15153-42fc-4aa0-be13-6a3d28ada157 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.171226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.171774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.173116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.173876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.174221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.175691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:15.176200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.178809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.207122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:15.219715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 903/1829] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:15.221016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.224976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 927/1830] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:15.232031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.239245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5d97c-0a5f-48ae-8c5c-240ee205a7c9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.323872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dbd3c-a3ef-4725-b506-c56e4ce972be admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.330501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.331222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.331591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.331877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.332573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.335222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.374265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 928/1831] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:15.378488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.384553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.419543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1d4d-ad4b-41fd-b863-ebc65b548087 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.436550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.437291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.440667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.440878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] PATH_INFO: `/v3/projects/b7d4afa45e92477eab2c1335b4953ebb` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.445265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d4afa45e92477eab2c1335b4953ebb)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.447264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.456245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.456999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.457782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.458307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.459934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.464722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.542095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 904/1832] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/projects => generated 365 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:15.544376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.622582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d4afa45e92477eab2c1335b4953ebb', 'request_id': 'req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:15.636604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d4afa45e92477eab2c1335b4953ebb', 'request_id': 'req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:15.647438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d4afa45e92477eab2c1335b4953ebb', 'request_id': 'req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:15.675711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] The token cache is being invalidate because project b7d4afa45e92477eab2c1335b4953ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:15.676752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:15.681626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:15.700869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 929/1833] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:15 2022] DELETE /identity/v3/projects/b7d4afa45e92477eab2c1335b4953ebb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:15.703380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:15.706516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:15.749582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576edb94-5db4-4d6c-8ba6-23fd11ebae6c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.838725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:15.919075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.919075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.919075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.919075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.919505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.919505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda8f2-0728-4ed6-87d2-5d3c8fa1e878 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.939239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbe9b9-b96b-473c-82bf-a08c176aaa7a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:15.946959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:15.948086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.948572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.949182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:15.949914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:15.952038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:15.975521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 905/1834] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:15.977722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:15.978044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:15.978351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.016624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 930/1835] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:16.017215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.024184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.038031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:16.040258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.058378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba254d-92b1-47ec-bf9a-6a603065022b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.072781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.073437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.073723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.073997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.074831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.085033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e1538-28be-4a2c-96ac-bd6f6cf7928e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.085270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 906/1836] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:16.085270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.109478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cce35-74be-4301-bd88-b73df81b3270 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.117674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 931/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:16.119209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.119500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.119775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.138040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.138929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.139502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.139928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.140724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.145874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.175260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 907/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:16.176271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.183941 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:16.190505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.195036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b6b83-40cc-4a6a-9747-28d15899445a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.216131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.216999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.217415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.217843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] PATH_INFO: `/v3/projects/fbf9474943254350b83b6ac99a07daff/users/80981791a14a47d191b1cb12ccc25ccf/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.231627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 932/1839] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:16.235429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:16.236418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.252171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf9474943254350b83b6ac99a07daff, user_id=80981791a14a47d191b1cb12ccc25ccf, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.255639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.268471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03d425-2ae4-4b7d-9652-23cf0c60119a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.281398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.282111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.282401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.282888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.284201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 908/1840] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:16 2022] PUT /identity/v3/projects/fbf9474943254350b83b6ac99a07daff/users/80981791a14a47d191b1cb12ccc25ccf/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:16.285476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.297035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.305333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8e925-e402-4291-b2cf-c63ae1a12d3d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.341827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 933/1841] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:16.343307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.343596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.343869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.430481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] MFA Rules not processed for user `80981791a14a47d191b1cb12ccc25ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:16.431778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.452301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae1e9-5b8c-4070-8bad-70ca20eb223c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.490479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.503277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.504056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.504958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.505492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:16.505967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.508180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.543324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 909/1842] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:16.543324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.558517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff927f7b-3e67-4683-bee0-7c7839454bee admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.576625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.578421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.578421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.578421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] PATH_INFO: `/v3/projects/0dbbf6b4a0a345da9815c69ae3da3d18/users/4880de63991f4ec3abb338b828ed3836/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.637126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbbf6b4a0a345da9815c69ae3da3d18, user_id=4880de63991f4ec3abb338b828ed3836, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.640050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.655098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 934/1843] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:16.655098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.662117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 910/1844] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:16 2022] PUT /identity/v3/projects/0dbbf6b4a0a345da9815c69ae3da3d18/users/4880de63991f4ec3abb338b828ed3836/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:16.673018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e40ad-e0da-40b0-b6a1-feb39e1da200 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.675306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.677171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.775160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.776940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.777750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.778243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.779935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.786037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.835905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54c9b5-4559-4064-9057-62825372f07d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.845460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 935/1845] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:16 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:16.847252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:16.851128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.851816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.852126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.852437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.858964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.872693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:16.892321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:16.893971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20975-24c0-4e26-b0b3-47ed7169a8fa service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:16.931649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 911/1846] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:16.931649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.931649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.931649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.974490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] MFA Rules not processed for user `4880de63991f4ec3abb338b828ed3836`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:16.977156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:16.979641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3bf1-8712-42d7-9a5b-b3711d48d6f6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:16.990043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:16.995137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:16.995433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:16.995706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:16.996545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.002024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.021112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.059635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 936/1847] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:16 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:17.059635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.073714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 912/1848] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:17.074721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.097118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6109-3aa5-4ab6-a589-4f66e31961ef None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.176982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aceca-f272-4b6e-ad44-fe4308c5c85a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.182703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.183278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.183560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.183827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.184643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.186579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.189024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.189800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.190176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.191554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.191554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.193405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.219123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 937/1849] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:17.220018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.231710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1963b-375c-4fc2-b19f-80004615b940 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.235364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 913/1850] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:17.236581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.238477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.239188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.239478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.239816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.240558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.243600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.256295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa30-b674-4744-b430-2f586b97fb55 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.264310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.265620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.265956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.269368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.270227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.272330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.286154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 938/1851] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:17.286991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.288838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 914/1852] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:17.289747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.302283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5eb5f6-90da-47cf-a068-fe99f5473cb1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.452213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1dfb37-ab54-4ad9-91a1-396c36d9d326 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.459720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.460655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.461290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.461617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.462404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.465018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.473260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.474841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.474841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.474841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.475343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.478082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.499105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 939/1853] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:17.500120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.511659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 915/1854] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:17.512547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.518407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.525631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48302e3f-d7d2-493a-a7a6-0391e0b1349d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.526238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcf162-2df4-434e-b194-720721350ec3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.532097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.534616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.534902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.535220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] PATH_INFO: `/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9/users/e3b2866db3c74f669b0dd7acc902a732/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.536674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.537065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.537484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.539702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.540633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.544814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.559929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 940/1855] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:17.560928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.565166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bf9b626a7943dcb8a2bb28b4212ee9, user_id=e3b2866db3c74f669b0dd7acc902a732, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.568050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.576491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef1e33-8331-430d-a48f-6f56c654d471 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.582627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.583763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.584319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.584692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.585539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.586791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 916/1856] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:17 2022] PUT /identity/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9/users/e3b2866db3c74f669b0dd7acc902a732/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:17.587477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.590804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.592839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.600778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 941/1857] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:17.601862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.614233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535d61-40ca-4460-ace4-c3450c16f33a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.620378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.677521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffe679-6cc1-4e07-957d-ca7e12d1cca7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.682572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.683255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.683532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.683802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] PATH_INFO: `/v3/users/6b42086b77a441159f439108cebd8dec` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.685032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b42086b77a441159f439108cebd8dec)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.686961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.702473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.703428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.703683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.704051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] PATH_INFO: `/v3/projects/884c59bd72be4d89b1cb1bf919d2bc5c/users/0d3029ca2aa0482a8f8ca428d7a77f7d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.725091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884c59bd72be4d89b1cb1bf919d2bc5c, user_id=0d3029ca2aa0482a8f8ca428d7a77f7d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.728616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.746194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b42086b77a441159f439108cebd8dec'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:17.752635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 942/1858] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:17 2022] PUT /identity/v3/projects/884c59bd72be4d89b1cb1bf919d2bc5c/users/0d3029ca2aa0482a8f8ca428d7a77f7d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:17.753743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.757714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b42086b77a441159f439108cebd8dec'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:17.775502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b42086b77a441159f439108cebd8dec'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:17.776350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b42086b77a441159f439108cebd8dec'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:17.791280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b42086b77a441159f439108cebd8dec'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:17.798690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 917/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:17 2022] DELETE /identity/v3/users/6b42086b77a441159f439108cebd8dec => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:17.802408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:17.804146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.829039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf2f5-48e8-46bc-b3a5-943144456223 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.885085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8ac8-a297-42fd-bf1c-28f94df795c6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:17.891712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.892426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.892704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.893042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] PATH_INFO: `/v3/projects/c0a861bf02294e9189fbee4549e0521b/users/b35a697c92ee484096ba7b735ea86d23/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.905449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:17.906072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.906345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.906613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.907300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.908994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51a2a-9465-4d9f-b03f-06e13e002cd4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.909480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a861bf02294e9189fbee4549e0521b, user_id=b35a697c92ee484096ba7b735ea86d23, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:17.912371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17719d-2e72-4d93-9589-331c38c01a54 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:17.923700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 943/1860] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:17.923700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215107d6-4792-4541-82b4-391aad021566 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.923700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215107d6-4792-4541-82b4-391aad021566 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.923700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215107d6-4792-4541-82b4-391aad021566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.935418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 918/1861] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:17 2022] PUT /identity/v3/projects/c0a861bf02294e9189fbee4549e0521b/users/b35a697c92ee484096ba7b735ea86d23/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:17.936770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:17.937048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:17.937326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:17.942057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:17.965912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-215107d6-4792-4541-82b4-391aad021566 None None] MFA Rules not processed for user `0d3029ca2aa0482a8f8ca428d7a77f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:17.967387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-215107d6-4792-4541-82b4-391aad021566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.971756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:17.974441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:17.987985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:18.040485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 944/1862] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:18.041496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215107d6-4792-4541-82b4-391aad021566 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.042882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-215107d6-4792-4541-82b4-391aad021566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.058846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 919/1863] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:18.058846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.064473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215107d6-4792-4541-82b4-391aad021566 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.067667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24a0c-c37c-4312-95af-8bbe4721610b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.070661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.071543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.071890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.072209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] PATH_INFO: `/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9/users/e3b2866db3c74f669b0dd7acc902a732/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.073814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.074371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bf9b626a7943dcb8a2bb28b4212ee9, user_id=e3b2866db3c74f669b0dd7acc902a732, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.074699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.074913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.075288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.076007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.077602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.077723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.089865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 945/1864] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:18 2022] PUT /identity/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9/users/e3b2866db3c74f669b0dd7acc902a732/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:18.090833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.092526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.099581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 920/1865] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:18.101176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.103812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.163418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863977f-9bac-4927-8362-808f3d537173 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.167343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.167963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.168239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.168506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.174021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36a2aa-d6d5-4063-9f53-9fe52b7da1ec admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.231855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.232617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.232896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.233163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.233521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.234999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c640b34-a451-4ca2-a8f6-dd8b70e55f40 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.244316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 946/1866] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:18.245739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeaeb9-f784-4a17-a088-d7249b75e914 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.246026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeaeb9-f784-4a17-a088-d7249b75e914 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.246305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeaeb9-f784-4a17-a088-d7249b75e914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.274105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-bceeaeb9-f784-4a17-a088-d7249b75e914 None None] MFA Rules not processed for user `e3b2866db3c74f669b0dd7acc902a732`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:18.275385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bceeaeb9-f784-4a17-a088-d7249b75e914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.301441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:18.304900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.306597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.316456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 921/1867] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:18.317235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.327261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83add95a-883b-419e-8a53-98414dba9068 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.332057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.332743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.332998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.333268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] PATH_INFO: `/v3/projects/c0a861bf02294e9189fbee4549e0521b/users/b35a697c92ee484096ba7b735ea86d23/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.335292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a861bf02294e9189fbee4549e0521b, user_id=b35a697c92ee484096ba7b735ea86d23, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.338391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.345614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:18.351365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 922/1868] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:18 2022] PUT /identity/v3/projects/c0a861bf02294e9189fbee4549e0521b/users/b35a697c92ee484096ba7b735ea86d23/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:18.356358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 947/1869] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:18.356525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.356525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.362654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.363358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.364467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.400381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] MFA Rules not processed for user `b35a697c92ee484096ba7b735ea86d23`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:18.402078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.460256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24612836-6aec-4211-92a1-c7ab4f3279a9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.466739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.467489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.467771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.468048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] PATH_INFO: `/v3/users/0d3029ca2aa0482a8f8ca428d7a77f7d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.469410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3029ca2aa0482a8f8ca428d7a77f7d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.471473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.529963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 948/1870] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:18.531162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.535750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.541643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3029ca2aa0482a8f8ca428d7a77f7d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:18.553572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3029ca2aa0482a8f8ca428d7a77f7d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:18.564492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3029ca2aa0482a8f8ca428d7a77f7d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:18.565234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3029ca2aa0482a8f8ca428d7a77f7d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:18.583795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3029ca2aa0482a8f8ca428d7a77f7d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:18.596138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 923/1871] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:18 2022] DELETE /identity/v3/users/0d3029ca2aa0482a8f8ca428d7a77f7d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:18.596895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.598693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.636561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177254b1-82e3-4a2e-8d59-8adbfae2caed None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.673780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb3328-a7c9-4c76-8ec3-a72e86e8822b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.677983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.678345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.678628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.678910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.680780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.705654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.706286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.706568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.706840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.708664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.717968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:18.753685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:18.759517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.761888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c72772-f306-4786-947c-c323bb1255ce service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.776538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 924/1872] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:18.776538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.776538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.776538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.798276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:18.800273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.825364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.827453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:18.844395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 949/1873] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:18.845277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.846627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.923411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 925/1874] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:18.925351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:18.947364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94bd24-af21-42bf-8bc0-ed1b5aa6dc5a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.961364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.963781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.964253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.964749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.966689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:18.971238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ee9c5-433c-427f-8490-b040a69e80f3 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:18.976800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:18.977703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:18.977974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:18.978264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] PATH_INFO: `/v3/projects/0f39010f618c467993f15da835cb51ed` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:18.979685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f39010f618c467993f15da835cb51ed)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:18.982277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.043937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:19.044880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.047586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f39010f618c467993f15da835cb51ed', 'request_id': 'req-62a56366-3e6c-48ad-9134-c90a2fdb531f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.054766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 926/1875] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:19.055235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f39010f618c467993f15da835cb51ed', 'request_id': 'req-62a56366-3e6c-48ad-9134-c90a2fdb531f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.055766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:19.057249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.062778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f39010f618c467993f15da835cb51ed', 'request_id': 'req-62a56366-3e6c-48ad-9134-c90a2fdb531f'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.087612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] The token cache is being invalidate because project 0f39010f618c467993f15da835cb51ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:19.088359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.088578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.094896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 950/1876] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:18 2022] DELETE /identity/v3/projects/0f39010f618c467993f15da835cb51ed => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:19.096401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:19.098410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.169288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfeca7-6be1-4dad-840d-45e04c1066a2 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:19.171814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:19.201685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:19.227478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a56366-3e6c-48ad-9134-c90a2fdb531f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:19.245652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:19.246388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:19.246671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:19.247040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] PATH_INFO: `/v3/users/58c0f874cbd5464ea9565a810a989aae` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:19.249568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:19.249568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:19.249793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:19.249936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:19.250759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c0f874cbd5464ea9565a810a989aae)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:19.251969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.253032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.269132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:19.269557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.280857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 927/1877] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:19.363789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c0f874cbd5464ea9565a810a989aae'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.375798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c0f874cbd5464ea9565a810a989aae'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.376873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c0f874cbd5464ea9565a810a989aae'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.401448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c0f874cbd5464ea9565a810a989aae'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.418366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c65a746c-6bd6-4b63-bcc9-b59f754f06ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c0f874cbd5464ea9565a810a989aae'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.429656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 951/1878] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:19 2022] DELETE /identity/v3/users/58c0f874cbd5464ea9565a810a989aae => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:19.599280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:19.601127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.613257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbbd6-fc84-4d68-8f28-9d51aa855e5d service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:19.676450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:19.677138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:19.677422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:19.677695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] PATH_INFO: `/v3/users/80981791a14a47d191b1cb12ccc25ccf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:19.679070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80981791a14a47d191b1cb12ccc25ccf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:19.681217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.738068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80981791a14a47d191b1cb12ccc25ccf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.746208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80981791a14a47d191b1cb12ccc25ccf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.755628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80981791a14a47d191b1cb12ccc25ccf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.756659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80981791a14a47d191b1cb12ccc25ccf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.774718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80981791a14a47d191b1cb12ccc25ccf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:19.781776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 928/1879] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:19 2022] DELETE /identity/v3/users/80981791a14a47d191b1cb12ccc25ccf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:19.786563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:19.787314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:19.787652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:19.804592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:19.806299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.809042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:19.810640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.872720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:19.897217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 952/1880] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:19.908792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:19.910606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:19.922043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595106dc-33dd-4dd8-9e67-593d63840094 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:19.928432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:19.929863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:19.930148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:19.930428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] PATH_INFO: `/v3/projects/884c59bd72be4d89b1cb1bf919d2bc5c` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:19.931827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884c59bd72be4d89b1cb1bf919d2bc5c)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:19.934403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:19.995635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883dc5c-e16b-4e62-845e-6580735c0b25 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.000901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884c59bd72be4d89b1cb1bf919d2bc5c', 'request_id': 'req-3f74340b-8261-448f-b16a-e866723c2382'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.007951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884c59bd72be4d89b1cb1bf919d2bc5c', 'request_id': 'req-3f74340b-8261-448f-b16a-e866723c2382'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.018036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:20.020148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884c59bd72be4d89b1cb1bf919d2bc5c', 'request_id': 'req-3f74340b-8261-448f-b16a-e866723c2382'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.039946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] The token cache is being invalidate because project 884c59bd72be4d89b1cb1bf919d2bc5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:20.040262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.040932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3f74340b-8261-448f-b16a-e866723c2382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.042701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 929/1881] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:19 2022] DELETE /identity/v3/projects/884c59bd72be4d89b1cb1bf919d2bc5c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:20.047705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.048505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.048947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.055715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.056435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.056871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.057209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.059687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.070186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.071235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.125428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.126894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb39b-2d1f-4ec6-9967-d29f4baaa577 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.140204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 953/1882] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:20.146223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 930/1883] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:20.154134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.154420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.154693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.182428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.183602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.187558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.189748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.196715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 954/1884] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:20.201586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.207084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a615-b4aa-4441-bfe8-262df94cd14a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.213980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.214171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ce8fd-d0c0-43e4-a4fe-89c811bf7744 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.215489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.216033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.216538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] PATH_INFO: `/v3/projects/df726c0a470144ea8b3490aadef8dc7b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.218110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df726c0a470144ea8b3490aadef8dc7b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.220452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.221096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.221434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.221723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.221994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.222745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:20.223188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.225130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.261763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 955/1885] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:20.262895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.277446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121424bb-ea92-43c8-9951-6d95cc9abf1c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.283808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.284437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.284713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.285073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.285624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.288195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.313372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df726c0a470144ea8b3490aadef8dc7b', 'request_id': 'req-0f2193b3-0d20-4826-be6d-35bf2088e1b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.322397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df726c0a470144ea8b3490aadef8dc7b', 'request_id': 'req-0f2193b3-0d20-4826-be6d-35bf2088e1b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.331111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 956/1886] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:20.336007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df726c0a470144ea8b3490aadef8dc7b', 'request_id': 'req-0f2193b3-0d20-4826-be6d-35bf2088e1b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.337403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.348712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:20.372671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] The token cache is being invalidate because project df726c0a470144ea8b3490aadef8dc7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:20.374347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.375709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f2193b3-0d20-4826-be6d-35bf2088e1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:20.383990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 931/1887] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:20 2022] DELETE /identity/v3/projects/df726c0a470144ea8b3490aadef8dc7b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:20.385806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933273-ab93-4ab7-922d-42899c8c31f9 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.386188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933273-ab93-4ab7-922d-42899c8c31f9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.386635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933273-ab93-4ab7-922d-42899c8c31f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.484071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-cb933273-ab93-4ab7-922d-42899c8c31f9 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.485416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cb933273-ab93-4ab7-922d-42899c8c31f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.499332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e907b9-f079-45da-ae87-3b8911613bdb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.505176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.506335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.506776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.507816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.509013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.512395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.517680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 932/1888] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:20.526698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.527200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.527634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.558253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.559601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 957/1889] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:20.559738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.563986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.566184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.575068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 933/1890] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:20.580588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.583952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07f54-4be0-4c49-b0e6-648d93fc6628 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.589987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.590609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.590952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.591976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.592110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.594050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.595236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fe145-695d-49f7-ac61-49f78c37aef0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.603709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.606582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.606582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.606582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.606582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:20.606582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.609101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.613809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 958/1891] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:20.615096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.624288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 934/1892] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:20.628828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.630695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20099c99-4372-44af-95f5-9257445b9e72 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.639079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.639719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.639995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.640263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.640970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.646309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.648030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:20.650385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54b22-58ef-4ddc-bf59-202a6f9957b0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.664955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.666852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.667433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.668859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 959/1893] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:20.669732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.669852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.674442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.677626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.699159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a426-fe70-4eb8-a6c9-17772659d03d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.708399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:20.709279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.709562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.709837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] PATH_INFO: `/v3/projects/2e2996451c004a6fb8221cc1aa60011b/users/42436de9093842afa6deed0f52f1a91d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.727762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2996451c004a6fb8221cc1aa60011b, user_id=42436de9093842afa6deed0f52f1a91d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:20.729281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 935/1894] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:20.730546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4d84-3654-47e6-9073-bf9792a20158 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:20.733654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:20.776140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 960/1895] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:20 2022] PUT /identity/v3/projects/2e2996451c004a6fb8221cc1aa60011b/users/42436de9093842afa6deed0f52f1a91d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:20.777814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:20.778103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:20.778435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:20.845861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] MFA Rules not processed for user `42436de9093842afa6deed0f52f1a91d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:20.847163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:20.871460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8420-4390-46b4-a61c-9ebb4eb74673 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:20.974630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 961/1896] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:20.974630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.023597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.024466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.025075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.025444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.026847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.029528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.072214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 936/1897] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/users => generated 286 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:21.073046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ebede-0d80-4ea8-b5dc-03a22172e2f4 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.073777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.079210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.080056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.080618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.080989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.082649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.085275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.085783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb3f12-cee4-4dcf-9435-fd2afd5b6702 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.092161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.093931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.094216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.094492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.095186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.096798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.117105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 937/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:21.122871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.135739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 962/1899] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:20 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:21.137805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7e1d5-1b4a-4180-a478-b9f15859ea5a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.140683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.205890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:21.238647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:21.255089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.256393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.257139 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.257675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.259252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.261519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.265848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc2831-c29b-4e1e-9b22-7863a8319a08 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.271735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.272562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.272843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.273116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.273759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.275741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.279675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 938/1900] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:21.283841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.310455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb800619-c883-4e97-91b7-3ab127790d8f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.312065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 963/1901] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:21 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:21.316068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.358179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.358179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.358179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.358622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] PATH_INFO: `/v3/projects/1f234be758ac476b82e14afbd62ba0e5/users/acedd2df5e604530a5cdff775054cf7e/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.368449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fab3-728f-4ce3-9f57-f2397a59b624 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.402787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.403639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.403930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.404199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.404876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.406466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.421093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f234be758ac476b82e14afbd62ba0e5, user_id=acedd2df5e604530a5cdff775054cf7e, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.426563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0351b0-089c-4536-933a-fb739a594cb2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.443876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 964/1902] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:21.445099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.462701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 939/1903] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:21 2022] PUT /identity/v3/projects/1f234be758ac476b82e14afbd62ba0e5/users/acedd2df5e604530a5cdff775054cf7e/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:21.467868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.467868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.468113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.473501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b146d-c0af-47c8-8cec-01ba5c197b5b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.519661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:21.521065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:21.592952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.593823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.594215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.594784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] PATH_INFO: `/v3/projects/285d8c65ba164cc2a92fb22e5c21048c/users/7e129e5f98444432b6c5df9b3f2d1555/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.616678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285d8c65ba164cc2a92fb22e5c21048c, user_id=7e129e5f98444432b6c5df9b3f2d1555, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.620931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.638554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 940/1904] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:21.641147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.644829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:21.648254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 965/1905] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:21 2022] PUT /identity/v3/projects/285d8c65ba164cc2a92fb22e5c21048c/users/7e129e5f98444432b6c5df9b3f2d1555/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:21.649811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.671452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb8d78-e38f-4b0e-8c4b-309e956e3256 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.758180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bb789-013f-4654-a269-79b43bb46051 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.763671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.766009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.767150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.767571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.768351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.770990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.790232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 966/1906] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:21.793737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:21.797875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:21.824558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:21.826089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:21.827365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:21.827910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] PATH_INFO: `/v3/projects/fbf9474943254350b83b6ac99a07daff` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:21.832682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf9474943254350b83b6ac99a07daff)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:21.836607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:21.920100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:21.926746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf9474943254350b83b6ac99a07daff', 'request_id': 'req-794176be-f91a-42b7-b4b5-a5fb4a9885c8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:21.930265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:21.933901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af73bc-adc7-4acf-8081-f27c951864f3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:21.943945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf9474943254350b83b6ac99a07daff', 'request_id': 'req-794176be-f91a-42b7-b4b5-a5fb4a9885c8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.001397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf9474943254350b83b6ac99a07daff', 'request_id': 'req-794176be-f91a-42b7-b4b5-a5fb4a9885c8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.019317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] The token cache is being invalidate because project fbf9474943254350b83b6ac99a07daff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:22.019664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.020440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.022633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 941/1907] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:21 2022] DELETE /identity/v3/projects/fbf9474943254350b83b6ac99a07daff => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:22.036847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:22.041364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.088936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.089514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.090923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.091238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.096641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.204754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794176be-f91a-42b7-b4b5-a5fb4a9885c8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:22.219722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.220673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.221020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.221823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] PATH_INFO: `/v3/projects/285d8c65ba164cc2a92fb22e5c21048c/users/7e129e5f98444432b6c5df9b3f2d1555/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.228104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285d8c65ba164cc2a92fb22e5c21048c, user_id=7e129e5f98444432b6c5df9b3f2d1555, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:22.232850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:22.260777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 942/1908] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:22 2022] PUT /identity/v3/projects/285d8c65ba164cc2a92fb22e5c21048c/users/7e129e5f98444432b6c5df9b3f2d1555/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:22.264477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:22.270241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.285553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:22.288049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e403b-1a2c-4767-a64b-e7f9fa37fbe3 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:22.300092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 967/1909] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:22.302645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be026-aa03-4006-b421-e615694e3802 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.303773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be026-aa03-4006-b421-e615694e3802 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.304252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be026-aa03-4006-b421-e615694e3802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.344373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-579be026-aa03-4006-b421-e615694e3802 None None] MFA Rules not processed for user `7e129e5f98444432b6c5df9b3f2d1555`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:22.345616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-579be026-aa03-4006-b421-e615694e3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.419335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699b41-1b41-452a-b9c3-fa10c9a062c4 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:22.432129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.432858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.433202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.433577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] PATH_INFO: `/v3/users/e3b2866db3c74f669b0dd7acc902a732` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.437413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b2866db3c74f669b0dd7acc902a732)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:22.439836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:22.506828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 968/1910] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:22.506828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be026-aa03-4006-b421-e615694e3802 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:22.525553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-579be026-aa03-4006-b421-e615694e3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.548644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be026-aa03-4006-b421-e615694e3802 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:22.552058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b2866db3c74f669b0dd7acc902a732'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.555140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.555918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.556203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.556486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] PATH_INFO: `/v3/users/4880de63991f4ec3abb338b828ed3836` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.557667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4880de63991f4ec3abb338b828ed3836)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:22.559674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:22.571369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b2866db3c74f669b0dd7acc902a732'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.586633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b2866db3c74f669b0dd7acc902a732'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.587397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b2866db3c74f669b0dd7acc902a732'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.609648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:22.614453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b2866db3c74f669b0dd7acc902a732'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.628453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 943/1911] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:22 2022] DELETE /identity/v3/users/e3b2866db3c74f669b0dd7acc902a732 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:22.632921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:22.638110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.644851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:22.681253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4880de63991f4ec3abb338b828ed3836'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.699194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4880de63991f4ec3abb338b828ed3836'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.699894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4880de63991f4ec3abb338b828ed3836'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.717714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4880de63991f4ec3abb338b828ed3836'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.734873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1bd60e11-57f9-497b-83c2-be5724a1befc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4880de63991f4ec3abb338b828ed3836'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:22.747317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 969/1912] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:22 2022] DELETE /identity/v3/users/4880de63991f4ec3abb338b828ed3836 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:22.749095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.749455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.749738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.782377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e18dc7-23f1-448c-8247-02b52545a112 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:22.795175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:22.796535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.889106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.889993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.890528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.891842 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:22.896036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 970/1913] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:22.896223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.915103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:22.916983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:22.992575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd88c8-8114-42d4-ad42-c3f3e57ba71b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:22.997332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:22.997985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:22.998232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:22.998500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:23.021228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:23.026476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:23.027258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:23.028961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef2a4-9380-4325-9c5c-d8a7d60049d7 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:23.051122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 971/1914] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:23.058013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 944/1915] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:23.713040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:23.714625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:23.745874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4163f-e64e-43fd-9ae6-eee12a9bac5c service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:23.759165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:23.761339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:23.761769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:23.762324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] PATH_INFO: `/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:23.764787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bf9b626a7943dcb8a2bb28b4212ee9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:23.767600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:23.853464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754784-a575-4739-a44b-6468db4e6669 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:23.854388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754784-a575-4739-a44b-6468db4e6669 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:23.854388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754784-a575-4739-a44b-6468db4e6669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:23.886182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:23.894410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bf9b626a7943dcb8a2bb28b4212ee9', 'request_id': 'req-04cdf9a9-6267-41b5-87d2-dde7de9c0604'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:23.901441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7f754784-a575-4739-a44b-6468db4e6669 None None] MFA Rules not processed for user `acedd2df5e604530a5cdff775054cf7e`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:23.902893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7f754784-a575-4739-a44b-6468db4e6669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:23.904363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bf9b626a7943dcb8a2bb28b4212ee9', 'request_id': 'req-04cdf9a9-6267-41b5-87d2-dde7de9c0604'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:23.912228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bf9b626a7943dcb8a2bb28b4212ee9', 'request_id': 'req-04cdf9a9-6267-41b5-87d2-dde7de9c0604'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:23.931688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] The token cache is being invalidate because project b1bf9b626a7943dcb8a2bb28b4212ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:23.931893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:23.933620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:23.935689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 972/1916] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:23 2022] DELETE /identity/v3/projects/b1bf9b626a7943dcb8a2bb28b4212ee9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:23.936470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:23.938084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:23.974824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:23.997814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 945/1917] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:24.000090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.000449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.000794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.032165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:24.033472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.067826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdf9a9-6267-41b5-87d2-dde7de9c0604 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.075787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.076792 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.077185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.078071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] PATH_INFO: `/v3/projects/0dbbf6b4a0a345da9815c69ae3da3d18` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.080394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbbf6b4a0a345da9815c69ae3da3d18)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.082626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.156907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbbf6b4a0a345da9815c69ae3da3d18', 'request_id': 'req-27edb7ae-762b-4061-9b12-9610f2d88527'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.157113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 946/1918] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:24.158215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.159847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.166327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbbf6b4a0a345da9815c69ae3da3d18', 'request_id': 'req-27edb7ae-762b-4061-9b12-9610f2d88527'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.175137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbbf6b4a0a345da9815c69ae3da3d18', 'request_id': 'req-27edb7ae-762b-4061-9b12-9610f2d88527'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.176589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15395f0-96c6-46d3-90e8-08e858a86e88 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.200802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] The token cache is being invalidate because project 0dbbf6b4a0a345da9815c69ae3da3d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:24.201165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.201931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-27edb7ae-762b-4061-9b12-9610f2d88527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.207370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 973/1919] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:23 2022] DELETE /identity/v3/projects/0dbbf6b4a0a345da9815c69ae3da3d18 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:24.209372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.209849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.210216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.238606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:24.240253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.278236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.278986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.279302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.279580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.280297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.282879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.326530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 947/1920] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:24 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:24.326530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.337569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.363492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 974/1921] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:24.365371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.368102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.381265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:24.391302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95831f09-eb18-4a5a-83d5-b15ceaed38e5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.446988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682a96-5093-423a-a2ae-fb8136d2e949 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.453189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.454044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.454843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.454843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] PATH_INFO: `/v3/users/b35a697c92ee484096ba7b735ea86d23` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.456282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35a697c92ee484096ba7b735ea86d23)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.471377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.481552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.483036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.485067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.488232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.489150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.494113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.500263 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:24.532812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 975/1922] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:24 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:24.536317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35a697c92ee484096ba7b735ea86d23'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.537851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.547629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35a697c92ee484096ba7b735ea86d23'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.548833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ca0b4-6986-49cf-bd8e-41877452885b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.555590 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.556215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.556491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.556777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.557534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:24.557701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.559350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.560220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35a697c92ee484096ba7b735ea86d23'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.561270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35a697c92ee484096ba7b735ea86d23'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.577577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35a697c92ee484096ba7b735ea86d23'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:24.584320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 948/1923] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:24 2022] DELETE /identity/v3/users/b35a697c92ee484096ba7b735ea86d23 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:24.585874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.586529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 976/1924] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:24.588161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.595923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.597670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:24.613240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030f43-19e3-4ab1-9ad8-9541b91656a5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.694622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.695664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.696200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.696748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.697550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.700091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.715489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd78bc3-9331-4351-b53e-39db6bb61d82 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.723226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.724230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.725044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.725591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.726349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.729022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.737639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 977/1925] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:24 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:24.742599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.747615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 949/1926] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:24.751884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.767651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077547e9-4392-4589-a4df-ef9a90c5191d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.852397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80458d-93fd-466f-9f74-0dd7edd33a83 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.854323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.855102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.855379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.855679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] PATH_INFO: `/v3/projects/fd4796ba69e747f580182e09bdb18294/users/d670cf378fdf4da69cb5ebedd1fa5473/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.858426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.859865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.859865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.859865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.860561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.862617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.874227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4796ba69e747f580182e09bdb18294, user_id=d670cf378fdf4da69cb5ebedd1fa5473, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.877078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:24.878319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:24.895284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 950/1927] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:24 2022] PUT /identity/v3/projects/fd4796ba69e747f580182e09bdb18294/users/d670cf378fdf4da69cb5ebedd1fa5473/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:24.895807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 978/1928] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:24 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:24.898623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.899614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:24.915497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1fcd0-31e1-42bd-b8f0-6323678d138f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.920496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:24.994590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf359fd-15c2-44d5-8233-a3a21b1cc064 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:24.995536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:24.996158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:24.996433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:24.996772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:24.997369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:24.998992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.002501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.003180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.003463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.003738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.004387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.007156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.012278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 979/1929] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:25.016402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.021880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 951/1930] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:25.023785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.028240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb07bc1-d20f-4ac1-8a97-88213689e854 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.034007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.034318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.034943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.034943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] PATH_INFO: `/v3/projects/fd4796ba69e747f580182e09bdb18294/users/d670cf378fdf4da69cb5ebedd1fa5473/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.037384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4796ba69e747f580182e09bdb18294, user_id=d670cf378fdf4da69cb5ebedd1fa5473, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.039581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4a9c1-0778-4ecd-9cea-6ae2e36e841c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.040584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.050119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.050989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.051296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.051632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.052477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.054744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21041f3b-ad02-4022-993a-29463146f07c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.058603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 980/1931] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:25 2022] PUT /identity/v3/projects/fd4796ba69e747f580182e09bdb18294/users/d670cf378fdf4da69cb5ebedd1fa5473/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:25.074847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 952/1932] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:25.076432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.150864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684db0b5-c353-4a8f-8c1b-416298ee97eb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.155592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.156274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.156548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.156818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] PATH_INFO: `/v3/projects/c1d156c427ec4d3a856662ea9dd61926/users/675806b6b05a4b1aaa9972c3d008dca6/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.174995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d156c427ec4d3a856662ea9dd61926, user_id=675806b6b05a4b1aaa9972c3d008dca6, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.178588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.195245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 981/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:25 2022] PUT /identity/v3/projects/c1d156c427ec4d3a856662ea9dd61926/users/675806b6b05a4b1aaa9972c3d008dca6/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:25.205919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.206203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.206477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.241974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] MFA Rules not processed for user `675806b6b05a4b1aaa9972c3d008dca6`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:25.243189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:25.342055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 953/1934] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:25.346292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.425838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03bfd-ed44-42a2-ad4e-6b5c43d3649a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.439878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.440577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.440854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.441360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.442028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.444595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.454855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:25.496400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 982/1935] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:25 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:25.500944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.514296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d894-6493-47d0-a5fe-8efeb849fb70 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.537281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.539246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:25.732010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.733833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.734649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.736858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.738186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.745891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.774962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330b69f-1639-44c2-aaf7-926661969563 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.784825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.785634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.785980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.786513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] PATH_INFO: `/v3/projects/c0a861bf02294e9189fbee4549e0521b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.788508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a861bf02294e9189fbee4549e0521b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.796309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.799943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 954/1936] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:25 2022] POST /identity/v3/users => generated 298 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:25.807208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.832011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8767984-b58f-4158-b889-60f0fd6e81e8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.845942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:25.846894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:25.848349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:25.849867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:25.851764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:25.856034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:25.882300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 955/1937] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:25.883438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.887172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:25.903679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5ac26-8166-4b4f-afc7-941fb3dfcf59 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:25.915884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:25.922583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a861bf02294e9189fbee4549e0521b', 'request_id': 'req-51d0655c-06fc-4680-95b6-0184a0c37292'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:25.935871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a861bf02294e9189fbee4549e0521b', 'request_id': 'req-51d0655c-06fc-4680-95b6-0184a0c37292'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:25.951270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a861bf02294e9189fbee4549e0521b', 'request_id': 'req-51d0655c-06fc-4680-95b6-0184a0c37292'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:25.970512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] The token cache is being invalidate because project c0a861bf02294e9189fbee4549e0521b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:25.971228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:25.972116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:25.974558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 983/1938] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:25 2022] DELETE /identity/v3/projects/c0a861bf02294e9189fbee4549e0521b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:25.976546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:25.979835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.000672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.001461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.001808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.002145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] PATH_INFO: `/v3/users/42436de9093842afa6deed0f52f1a91d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.004235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42436de9093842afa6deed0f52f1a91d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.007098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.072104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42436de9093842afa6deed0f52f1a91d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:26.077564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0655c-06fc-4680-95b6-0184a0c37292 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.080836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42436de9093842afa6deed0f52f1a91d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:26.085256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.086109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.086464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.086936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] PATH_INFO: `/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa/users/5163fb1417ea41bc8bfa81d265e31859/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.093851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42436de9093842afa6deed0f52f1a91d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:26.095733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42436de9093842afa6deed0f52f1a91d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:26.112476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8ecb7599-90ef-482d-8e5b-f96d242e436b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42436de9093842afa6deed0f52f1a91d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:26.120620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 956/1939] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:25 2022] DELETE /identity/v3/users/42436de9093842afa6deed0f52f1a91d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:26.122201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298a51d-023b-4191-9a3d-ef960cd777e1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.122576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298a51d-023b-4191-9a3d-ef960cd777e1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.122961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298a51d-023b-4191-9a3d-ef960cd777e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.132144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e1c92c67e9427d8fc17bde957f7baa, user_id=5163fb1417ea41bc8bfa81d265e31859, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.153165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383910d-9939-4162-9316-02e1d8b1e04d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.155841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3298a51d-023b-4191-9a3d-ef960cd777e1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.158519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3298a51d-023b-4191-9a3d-ef960cd777e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.183876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:26.193124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 984/1940] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:25 2022] PUT /identity/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa/users/5163fb1417ea41bc8bfa81d265e31859/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:26.198419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811b43-a385-4b49-9f05-aee7263380f6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.198711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811b43-a385-4b49-9f05-aee7263380f6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.199764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811b43-a385-4b49-9f05-aee7263380f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.252063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 957/1941] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.253525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-cf811b43-a385-4b49-9f05-aee7263380f6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.254790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cf811b43-a385-4b49-9f05-aee7263380f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.260500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.261013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.261322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.265611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 958/1942] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:56:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:26.265611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.266754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.332353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:26.373143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 985/1943] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.375377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6253e-882b-4a1a-9a8f-2089e698138d None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.375890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6253e-882b-4a1a-9a8f-2089e698138d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.377713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fced3-44da-48b4-8403-58ba36bdbbb4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.384646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.384906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6253e-882b-4a1a-9a8f-2089e698138d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.385697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.386115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.386569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.387387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.389240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488dc03-f2a6-4602-999f-1f88bf6ff625 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.422319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-15a6253e-882b-4a1a-9a8f-2089e698138d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.425367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-15a6253e-882b-4a1a-9a8f-2089e698138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.440991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 959/1944] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:26.440991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.440991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.440991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.461328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 986/1945] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.463171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.463654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.464233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.477286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] MFA Rules not processed for user `d670cf378fdf4da69cb5ebedd1fa5473`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.478324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.489644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.491166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.560521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 960/1946] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.561476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.562813 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.585491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 987/1947] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.586444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.597317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c225d73-91f0-4d70-86f4-fc784b6dea92 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.603542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.604241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.604516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.604790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] PATH_INFO: `/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa/users/5163fb1417ea41bc8bfa81d265e31859/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.607400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e1c92c67e9427d8fc17bde957f7baa, user_id=5163fb1417ea41bc8bfa81d265e31859, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.609914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.624326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 988/1948] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:26 2022] PUT /identity/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa/users/5163fb1417ea41bc8bfa81d265e31859/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:26.625079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.640368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ef26-b98f-43b7-9fae-b63c50df8eb5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.707108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.707795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.708083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.708360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.710347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd532161-540f-4cc8-9c42-410acc5ee446 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.714984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.715856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.716196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.716525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.717245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:26.717608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.719354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac595dc-536b-45d9-87b3-fa453e7d8e2e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.720028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.721650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.732567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 961/1949] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:26.733297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.734267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 989/1950] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:26.736405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.736696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.736970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.738800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.754028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:26.757063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:26.778101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] MFA Rules not processed for user `5163fb1417ea41bc8bfa81d265e31859`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:26.779876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.819112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfd1-10f1-4bf7-bfa4-b3e2bbece530 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.823614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.824388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.824774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.825331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.828133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.909387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.910746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:26.928736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 962/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:26.944054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.948621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:26.957069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 990/1952] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:26.957069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:26.980295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a20f7-78d5-41a5-9098-493bf789caad None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:26.987694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:26.988738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:26.989326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:26.989818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:26.991530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:26.994415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.045213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 991/1953] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:26 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:27.046472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.049060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:27.148501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cef8-7a0b-4ffb-849b-9ae7fb008e1b service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.164585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6aed-13dc-4e1e-a8dc-512e581da39c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.170049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.189028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.189425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.189710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.192679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:27.294457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:27.313816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.315595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.315960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.316604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.316889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.317169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.330734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 992/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:27.331754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.358460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.360301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.372539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:27.424179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 963/1955] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:27.426093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.433162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:27.436340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a54ad-2a0d-4003-a5d4-52703cfee6b4 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.449305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.464115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.464115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.464115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.464817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.469169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.484553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fecd1-2861-459e-99d1-44ee86754ada service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.496810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.499283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.499742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.502865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] PATH_INFO: `/v3/projects/2e2996451c004a6fb8221cc1aa60011b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.504386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2996451c004a6fb8221cc1aa60011b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.509701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.515160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 993/1956] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:27 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:27.524139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.549364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae6194-4d5f-4cd7-be52-568b18f63537 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.576737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.577264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.577607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.577947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.579727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.584597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.613600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 994/1957] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:27.618263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.639247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f69027-270e-4575-95ce-9d584c93505e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.640693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2996451c004a6fb8221cc1aa60011b', 'request_id': 'req-fdc74dd7-34ed-413c-b3ef-37346a389d6c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.655857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2996451c004a6fb8221cc1aa60011b', 'request_id': 'req-fdc74dd7-34ed-413c-b3ef-37346a389d6c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.682178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2996451c004a6fb8221cc1aa60011b', 'request_id': 'req-fdc74dd7-34ed-413c-b3ef-37346a389d6c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.719121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] The token cache is being invalidate because project 2e2996451c004a6fb8221cc1aa60011b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:27.719459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.721615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.728940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 964/1958] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:27 2022] DELETE /identity/v3/projects/2e2996451c004a6fb8221cc1aa60011b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:27.730156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.732275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:27.776110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.776932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.777386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.777899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.778736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.780650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.794177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 995/1959] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:27.798077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:27.801609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:27.823530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb239-23bb-4e5f-8a32-cf485dda26a5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.830864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.832183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.832314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc74dd7-34ed-413c-b3ef-37346a389d6c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:27.832416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.832416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] PATH_INFO: `/v3/projects/983035e96dd74f448ac73025b1cdf9a9/users/51d3f279bc504210b6fbfef56b51f665/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.850466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:27.851473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.851807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.853079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] PATH_INFO: `/v3/users/7e129e5f98444432b6c5df9b3f2d1555` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.853587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983035e96dd74f448ac73025b1cdf9a9, user_id=51d3f279bc504210b6fbfef56b51f665, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.857432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e129e5f98444432b6c5df9b3f2d1555)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:27.860222 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.862506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb8840-2178-4b3c-9e95-f28006171a96 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:27.892271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 996/1960] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:27 2022] PUT /identity/v3/projects/983035e96dd74f448ac73025b1cdf9a9/users/51d3f279bc504210b6fbfef56b51f665/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:27.920572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:27.927292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:27.930030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:27.980304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e129e5f98444432b6c5df9b3f2d1555'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:27.999737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e129e5f98444432b6c5df9b3f2d1555'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.017848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] MFA Rules not processed for user `51d3f279bc504210b6fbfef56b51f665`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:28.020207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.033670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e129e5f98444432b6c5df9b3f2d1555'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.034382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e129e5f98444432b6c5df9b3f2d1555'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.052766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:28.059587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e129e5f98444432b6c5df9b3f2d1555'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.073387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 965/1961] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:27 2022] DELETE /identity/v3/users/7e129e5f98444432b6c5df9b3f2d1555 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:28.149219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 997/1962] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:28.154329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.156022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.259313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab681-e65e-4c1c-a4bd-75ea373054d9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.264860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.266008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.266291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.266559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.267248 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.270841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.311521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 966/1963] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:28 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:28.313506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.341036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:28.435392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f13eb-997e-4a17-abaa-efe09523d20a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.500568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.502262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.530107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.530931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.531729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.532015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.532661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.535870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.565284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 998/1964] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:28 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:28.569843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.582235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711e7ab-6c19-4f51-915f-4c54f3d836e9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.587221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.587979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.588391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.588668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.589323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.591110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.603285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 999/1965] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:28.607615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.609398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18f31e-f271-439e-81e0-4fb5f5430eff admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.614068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.614925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.615334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.615693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] PATH_INFO: `/v3/projects/285d8c65ba164cc2a92fb22e5c21048c` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.616891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=285d8c65ba164cc2a92fb22e5c21048c)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.618940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.631974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdb5a6-9eb0-4949-9573-2714ecdc56d4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.637025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.637779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.638125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.638463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] PATH_INFO: `/v3/projects/fb3193843ea245ebbeafb6a63f0d5281/users/f90f7d957f624fb9b54f9682bf1200ed/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.656149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3193843ea245ebbeafb6a63f0d5281, user_id=f90f7d957f624fb9b54f9682bf1200ed, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.658922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.681264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285d8c65ba164cc2a92fb22e5c21048c', 'request_id': 'req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.681998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1000/1966] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:28 2022] PUT /identity/v3/projects/fb3193843ea245ebbeafb6a63f0d5281/users/f90f7d957f624fb9b54f9682bf1200ed/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:28.683054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.686432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.688030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285d8c65ba164cc2a92fb22e5c21048c', 'request_id': 'req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.697357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285d8c65ba164cc2a92fb22e5c21048c', 'request_id': 'req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.710329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] The token cache is being invalidate because project 285d8c65ba164cc2a92fb22e5c21048c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:28.710643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.711381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:28.715305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 967/1967] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:28 2022] DELETE /identity/v3/projects/285d8c65ba164cc2a92fb22e5c21048c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:28.716150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.720243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.731039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:28.774017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4264d-8e04-467f-a974-42a76845b14a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.783149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.783149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.783149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.783149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.785120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:28.812597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d995b6a-fbd6-4a58-891d-6fa00f857fa7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.815760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:28.868771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.870259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.880381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.881247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1001/1968] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:28.881369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.882149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.882432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.883293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.885075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01e12a-c056-4880-bf2c-4c6b8dc2c166 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.899093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 968/1969] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:28.901215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:28.913733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ceaed9-d466-4425-a783-32119a302c8a service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:28.919990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:28.920767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.921045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.921321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] PATH_INFO: `/v3/projects/fb3193843ea245ebbeafb6a63f0d5281/users/f90f7d957f624fb9b54f9682bf1200ed/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.923497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3193843ea245ebbeafb6a63f0d5281, user_id=f90f7d957f624fb9b54f9682bf1200ed, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:28.926203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:28.939705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1002/1970] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:28 2022] PUT /identity/v3/projects/fb3193843ea245ebbeafb6a63f0d5281/users/f90f7d957f624fb9b54f9682bf1200ed/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:28.950816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:28.951217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:28.951497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:28.982493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] MFA Rules not processed for user `f90f7d957f624fb9b54f9682bf1200ed`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:28.983749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:29.062197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 969/1971] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:29.107226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:29.109106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:29.190585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089681c0-0006-4466-8369-bc810972b90e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:29.195585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:29.197098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:29.197525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:29.197796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:29.214278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:29.214278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5f389-0c31-4f9b-8f66-a259a5db084c service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:29.222367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1003/1972] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:31.108345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:31.157713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:31.186406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb6f2f-0da1-42aa-ad30-ea83526c1d0d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:31.190989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:31.191652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:31.191933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:31.192206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] PATH_INFO: `/v3/users/51d3f279bc504210b6fbfef56b51f665` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:31.194066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d3f279bc504210b6fbfef56b51f665)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:31.196173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:31.246831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d3f279bc504210b6fbfef56b51f665'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.254075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d3f279bc504210b6fbfef56b51f665'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.263934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d3f279bc504210b6fbfef56b51f665'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.264702 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d3f279bc504210b6fbfef56b51f665'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.277460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d3f279bc504210b6fbfef56b51f665'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.283575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 970/1973] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:31 2022] DELETE /identity/v3/users/51d3f279bc504210b6fbfef56b51f665 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:31.288709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:31.289306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:31.289306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:31.308759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:31.309970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:31.370931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1004/1974] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:31.381501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:31.382973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:31.446772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b651c-d163-444a-8af0-97b1cc8de4c2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:31.450542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:31.451194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:31.451470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:31.451740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:31.461950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:31.463587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:31.474431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 971/1975] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:31.782946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:31.784417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:31.799932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bcc66-c980-4001-bb4f-88a165edb51b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:31.807176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:31.807850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:31.808330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:31.808800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] PATH_INFO: `/v3/projects/983035e96dd74f448ac73025b1cdf9a9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:31.810393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983035e96dd74f448ac73025b1cdf9a9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:31.812388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:31.860907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:31.868939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983035e96dd74f448ac73025b1cdf9a9', 'request_id': 'req-dde22bf1-036b-480e-935f-237650f916b3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.877248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983035e96dd74f448ac73025b1cdf9a9', 'request_id': 'req-dde22bf1-036b-480e-935f-237650f916b3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.884128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983035e96dd74f448ac73025b1cdf9a9', 'request_id': 'req-dde22bf1-036b-480e-935f-237650f916b3'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.898173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] The token cache is being invalidate because project 983035e96dd74f448ac73025b1cdf9a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:31.898465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.899118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:31.900718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1005/1976] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:31 2022] DELETE /identity/v3/projects/983035e96dd74f448ac73025b1cdf9a9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:31.905015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:31.915291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b5-264d-4544-99c6-b746f6ed7b1e service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:31.987641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:31.994595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:31.996168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:31.997199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:31.997503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] PATH_INFO: `/v3/users/f90f7d957f624fb9b54f9682bf1200ed` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:31.998982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90f7d957f624fb9b54f9682bf1200ed)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:32.002645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:32.068903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90f7d957f624fb9b54f9682bf1200ed'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.078161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90f7d957f624fb9b54f9682bf1200ed'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.090615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90f7d957f624fb9b54f9682bf1200ed'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.091492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90f7d957f624fb9b54f9682bf1200ed'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.115600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90f7d957f624fb9b54f9682bf1200ed'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.119052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 972/1977] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:31 2022] DELETE /identity/v3/users/f90f7d957f624fb9b54f9682bf1200ed => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:32.258167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:32.259664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:32.326557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde22bf1-036b-480e-935f-237650f916b3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:32.391986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:32.392650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.392928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.393196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.395489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:32.439144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:32.440597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:32.487619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:32.489516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bf17b-0a74-4087-89ca-bcfd7e1c87bc service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:32.500880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1006/1978] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:32.545542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3d283-1379-49c9-b155-0b3028639fa3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:32.565107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:32.565810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.566092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.566364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] PATH_INFO: `/v3/projects/fb3193843ea245ebbeafb6a63f0d5281` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.567945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3193843ea245ebbeafb6a63f0d5281)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:32.570508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:32.633760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3193843ea245ebbeafb6a63f0d5281', 'request_id': 'req-3b8a274c-f19e-4022-9cd5-9f0236af8108'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.640007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3193843ea245ebbeafb6a63f0d5281', 'request_id': 'req-3b8a274c-f19e-4022-9cd5-9f0236af8108'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.652375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3193843ea245ebbeafb6a63f0d5281', 'request_id': 'req-3b8a274c-f19e-4022-9cd5-9f0236af8108'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.667429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] The token cache is being invalidate because project fb3193843ea245ebbeafb6a63f0d5281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:32.667772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.668513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3b8a274c-f19e-4022-9cd5-9f0236af8108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:32.670498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 973/1979] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:32 2022] DELETE /identity/v3/projects/fb3193843ea245ebbeafb6a63f0d5281 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:32.675519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.675709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.676047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.699903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:32.701333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:32.741951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:32.799388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1007/1980] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:32.808765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.809052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.809329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.837425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:32.845835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:32.848456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:32.866366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 974/1981] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:32.867863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:32.883483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc1529-eff8-4d21-831c-c0475493354e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:32.892176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:32.892931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.893289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.893690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.894489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:32.894791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:32.896535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:32.924210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:32.925144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1008/1982] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:32.952507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608586d-337c-427f-8d6e-c5e3f32e97ba None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:32.966859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:32.967544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:32.967819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:32.968241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:32.968893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:32.994507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.046273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 975/1983] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:32 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:33.050700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.063050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64385d-31cf-45ec-89aa-4fe3df8caf85 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.138416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.139800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.140083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.140349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.140989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.143380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.173793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1009/1984] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:33 2022] POST /identity/v3/users => generated 304 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:33.175481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.189053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65332e-e2ed-44c9-9ba8-76d33ee2e2b5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.196890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.197572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.197851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.198123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.198818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.200571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.225740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 976/1985] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:33.228881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.241570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3fefc-2ba7-4703-b274-d981fc977d58 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.247153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.247981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.248318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.248646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.249423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.251356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9f92a-e095-45d1-8e96-8edf468674c1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.284807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1010/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:33.289106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.306205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609acbd-8ed9-494d-bc6d-e54fa45d3013 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.312913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.313685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.314078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.314429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] PATH_INFO: `/v3/projects/b65e6d93672b46e19205544946e4f685/users/8799b869041640379a2cbd9202456314/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.338110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65e6d93672b46e19205544946e4f685, user_id=8799b869041640379a2cbd9202456314, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.341775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.387297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 977/1987] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:33 2022] PUT /identity/v3/projects/b65e6d93672b46e19205544946e4f685/users/8799b869041640379a2cbd9202456314/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:33.393135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.393614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.393691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.425109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] MFA Rules not processed for user `8799b869041640379a2cbd9202456314`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:33.426494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:33.516651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1011/1988] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:33 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:33.522692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.643408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3adec-b2a5-437e-af72-2d18caaa4cdb admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.649219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.650136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.650249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.650437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.651141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.653219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.687087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 978/1989] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:33 2022] POST /identity/v3/projects => generated 409 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:33.689994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.703577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfb3a9-5f7b-4ed1-81dc-7e4515acb325 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.740963 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:33.775770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.776288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.776567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.777998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.777998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.779470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.803993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1012/1990] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:33 2022] POST /identity/v3/users => generated 311 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:33.808238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.823215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a998e3-4661-4781-8527-5b5ca87b5fa1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.830237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.830863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.831180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.831579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.832351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.834288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.850419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 979/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:33.852163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.863371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e6e7b-8cd1-4766-9c11-3cf91a244efe admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:33.869514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:33.870926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:33.871244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:33.871516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] PATH_INFO: `/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6/users/0a81e13bded94338a9df6252230089da/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:33.890791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2f29a58e5d47a68b3492e30cde74b6, user_id=0a81e13bded94338a9df6252230089da, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:33.893698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:33.912079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1013/1992] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:33 2022] PUT /identity/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6/users/0a81e13bded94338a9df6252230089da/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:33.916242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:33.928431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b16b-0dec-4101-ab2b-d27026e93321 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:34.015110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:34.016169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:34.016581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:34.016986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:34.017774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:34.019551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549772-7bbf-4732-a2e9-a8687befa704 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:34.031661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 980/1993] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:34.035338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:34.051534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659ee47-0da7-4d3e-8ffd-3f5a4aee48cd admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:34.065207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:34.066287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:34.067027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:34.067436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] PATH_INFO: `/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6/users/0a81e13bded94338a9df6252230089da/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:34.072393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2f29a58e5d47a68b3492e30cde74b6, user_id=0a81e13bded94338a9df6252230089da, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:34.081229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:34.114278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1014/1994] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:34 2022] PUT /identity/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6/users/0a81e13bded94338a9df6252230089da/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:34.126280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:34.126570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:34.126861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:34.197281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:34.213078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] MFA Rules not processed for user `0a81e13bded94338a9df6252230089da`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:34.216663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:34.334317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 981/1995] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:34 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:34.361180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:34.362979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:34.506044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5b973-ca7b-44c5-9f12-185890e8768d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:34.517835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:34.519780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:34.520138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:34.521245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:34.548174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:34.550205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:34.577002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1015/1996] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:34 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:34.819167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:34.842486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8209d32-1b16-4f04-8844-3cdad0661515 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:34.846174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:34.846794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:34.847109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:34.847388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:34.956498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:34.958259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9784d-83db-48fe-85d9-77a9b02f45e9 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:34.975674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 982/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:34 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:37.899896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:37.901446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:37.976138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813a1c-c61d-4894-8fc7-73d9ae5e6a40 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:37.981133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f949f07-720d-4031-b417-777788483380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:37.981773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f949f07-720d-4031-b417-777788483380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:37.982053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f949f07-720d-4031-b417-777788483380 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:37.982327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f949f07-720d-4031-b417-777788483380 admin admin] PATH_INFO: `/v3/users/675806b6b05a4b1aaa9972c3d008dca6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:37.983556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f949f07-720d-4031-b417-777788483380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675806b6b05a4b1aaa9972c3d008dca6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:37.985507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f949f07-720d-4031-b417-777788483380 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:38.027102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:38.048686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675806b6b05a4b1aaa9972c3d008dca6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.056273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675806b6b05a4b1aaa9972c3d008dca6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.056995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675806b6b05a4b1aaa9972c3d008dca6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.070407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675806b6b05a4b1aaa9972c3d008dca6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.077818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675806b6b05a4b1aaa9972c3d008dca6'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.084296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1016/1998] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:37 2022] DELETE /identity/v3/users/675806b6b05a4b1aaa9972c3d008dca6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:38.088847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:38.089311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:38.089397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:38.109449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:38.110699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:38.181898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 983/1999] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:38.191865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:38.194486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:38.255891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f949f07-720d-4031-b417-777788483380 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:38.260387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:38.261008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:38.261285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:38.261554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:38.271550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:38.272996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:38.282990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1017/2000] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:38.575896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:38.577592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:38.593256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6ec6-8bc9-425c-9375-b055a6526170 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:38.598990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:38.599984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:38.600304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:38.600598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] PATH_INFO: `/v3/projects/c1d156c427ec4d3a856662ea9dd61926` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:38.601987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d156c427ec4d3a856662ea9dd61926)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:38.604009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:38.648641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:38.665039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d156c427ec4d3a856662ea9dd61926', 'request_id': 'req-dc0ce94c-71be-435b-8b8d-4dc05097d72c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.671473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d156c427ec4d3a856662ea9dd61926', 'request_id': 'req-dc0ce94c-71be-435b-8b8d-4dc05097d72c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.681008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d156c427ec4d3a856662ea9dd61926', 'request_id': 'req-dc0ce94c-71be-435b-8b8d-4dc05097d72c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.693681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] The token cache is being invalidate because project c1d156c427ec4d3a856662ea9dd61926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:38.693869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.694565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.696136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 984/2001] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:38 2022] DELETE /identity/v3/projects/c1d156c427ec4d3a856662ea9dd61926 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:38.700240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:38.710239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d17d5-cf74-4db2-b5a1-0ca822d8b31d service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:38.767795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:38.768432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:38.768708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:38.768974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] PATH_INFO: `/v3/users/5163fb1417ea41bc8bfa81d265e31859` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:38.770110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5163fb1417ea41bc8bfa81d265e31859)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:38.772100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:38.819907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5163fb1417ea41bc8bfa81d265e31859'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.821877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:38.828035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5163fb1417ea41bc8bfa81d265e31859'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.828946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5163fb1417ea41bc8bfa81d265e31859'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.842515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5163fb1417ea41bc8bfa81d265e31859'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.851974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c327559d-f7c3-4108-b32b-22ac85305af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5163fb1417ea41bc8bfa81d265e31859'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:38.858682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1018/2002] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:38 2022] DELETE /identity/v3/users/5163fb1417ea41bc8bfa81d265e31859 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:39.149612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.151282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:39.166141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ce94c-71be-435b-8b8d-4dc05097d72c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.240829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.241783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.242218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.242569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] PATH_INFO: `/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.244508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e1c92c67e9427d8fc17bde957f7baa)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.247272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.304114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e1c92c67e9427d8fc17bde957f7baa', 'request_id': 'req-c7837e33-7023-412b-8fea-1a789cc954e9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:39.313316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e1c92c67e9427d8fc17bde957f7baa', 'request_id': 'req-c7837e33-7023-412b-8fea-1a789cc954e9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:39.325185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e1c92c67e9427d8fc17bde957f7baa', 'request_id': 'req-c7837e33-7023-412b-8fea-1a789cc954e9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:39.336725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] The token cache is being invalidate because project d3e1c92c67e9427d8fc17bde957f7baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:39.337012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:39.337961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c7837e33-7023-412b-8fea-1a789cc954e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:39.339563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 985/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:39 2022] DELETE /identity/v3/projects/d3e1c92c67e9427d8fc17bde957f7baa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:39.345673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.346203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.346203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.366320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:39.367575 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:39.435180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1019/2004] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:39.445569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.445860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.446126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.467552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:39.468763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:39.477877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 986/2005] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:39.482199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.492768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf938b-b788-45c5-9b6b-811c99eaca3b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.498164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.498780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.499085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.499365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.499994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:39.500298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.501923 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.515248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1020/2006] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:39.520431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.530306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356090f-57c1-4036-8f85-48d9344fa48f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.535350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.535965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.536250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.536940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.537206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.539245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.572323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 987/2007] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:39 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:39.573490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.583921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febb1b-4a04-409c-83d9-ca36d36b75db admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.648221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.648836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.649114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.649383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.650058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.652096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.681014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1021/2008] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:39 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:39.685579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.697371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42d43-a60b-48bd-bf22-6511883499ce admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.703470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.704090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.704370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.704645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.705292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.706915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.718084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 988/2009] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:39.722291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.733417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c5c0-ed28-4c21-abb8-24de33d570fb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.738742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.739385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.739664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.739959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.741453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.742550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d14f5-d27f-496f-9c77-95405e8766c2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.753474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1022/2010] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:39.758070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.768868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bdc99-826a-4155-9a72-a327dc9d3d67 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.775205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.775895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.776174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.776447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] PATH_INFO: `/v3/projects/5b70056e81304145a833c88ea096e76d/users/309da81b8e13455a836e0c8435ed6b06/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.785461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:39.803626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b70056e81304145a833c88ea096e76d, user_id=309da81b8e13455a836e0c8435ed6b06, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:39.803626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:39.810934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 989/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:39 2022] PUT /identity/v3/projects/5b70056e81304145a833c88ea096e76d/users/309da81b8e13455a836e0c8435ed6b06/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:39.815879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef093b-ad1d-4098-949e-72260a81d855 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.816160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef093b-ad1d-4098-949e-72260a81d855 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.816426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef093b-ad1d-4098-949e-72260a81d855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:39.836550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-83ef093b-ad1d-4098-949e-72260a81d855 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:39.837749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-83ef093b-ad1d-4098-949e-72260a81d855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:39.868111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:39.905039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1023/2012] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:39.918354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:39.921380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:39.987827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1a200-35bf-4c96-9243-78260ce3dc5f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:39.992197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:39.992816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:39.993095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:39.993363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:40.002929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:40.004449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:40.014123 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 990/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:40.963385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82175b-4f4c-4d00-874b-a7c9ca6f2a56 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:40.963955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82175b-4f4c-4d00-874b-a7c9ca6f2a56 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:40.963955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82175b-4f4c-4d00-874b-a7c9ca6f2a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:41.012572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dc82175b-4f4c-4d00-874b-a7c9ca6f2a56 None None] MFA Rules not processed for user `309da81b8e13455a836e0c8435ed6b06`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:41.012572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dc82175b-4f4c-4d00-874b-a7c9ca6f2a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:41.113520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1024/2014] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:41.142985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:41.151712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:41.259117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305051-37a5-47c3-90ff-f7c67ea115a0 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:41.264096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:41.264914 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:41.265192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:41.265464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:41.275845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:41.277297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:41.287065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 991/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:42.705546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e35fd-ffab-4042-b596-84b4860a503a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:42.706048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e35fd-ffab-4042-b596-84b4860a503a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:42.706337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e35fd-ffab-4042-b596-84b4860a503a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:42.727972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-da3e35fd-ffab-4042-b596-84b4860a503a None None] MFA Rules not processed for user `309da81b8e13455a836e0c8435ed6b06`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:42.729236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-da3e35fd-ffab-4042-b596-84b4860a503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:42.738647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1025/2016] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:43.993423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:43.994932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.012654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239e7cc-9b26-40ab-b216-470be0339cac service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:44.019147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:44.019983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.020373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.020696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] PATH_INFO: `/v3/users/8799b869041640379a2cbd9202456314` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.022141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8799b869041640379a2cbd9202456314)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:44.024353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:44.151718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8799b869041640379a2cbd9202456314'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.163383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8799b869041640379a2cbd9202456314'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.178755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8799b869041640379a2cbd9202456314'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.180052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8799b869041640379a2cbd9202456314'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.198474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8799b869041640379a2cbd9202456314'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.210509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 992/2017] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:43 2022] DELETE /identity/v3/users/8799b869041640379a2cbd9202456314 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:44.212050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.212522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.212631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.236557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:44.238063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.316828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1026/2018] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:44.327344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:44.330732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.342424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:44.409043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6e38-c9fb-4593-9881-2a1b0675ee0a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:44.419096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:44.419744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.420023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.420297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.429911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:44.431485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d33a-2bc6-4dff-a6f1-1523f0e663c3 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:44.441617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 993/2019] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:44.737936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:44.739417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.751812 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e752-63e5-47b3-8821-9ae3c456ebfe None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:44.757834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:44.758538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.758800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.759269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] PATH_INFO: `/v3/projects/b65e6d93672b46e19205544946e4f685` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.760698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65e6d93672b46e19205544946e4f685)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:44.762721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:44.815437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65e6d93672b46e19205544946e4f685', 'request_id': 'req-1e82be15-1cde-40e2-9f32-1985ebd421ba'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.819617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:44.825844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65e6d93672b46e19205544946e4f685', 'request_id': 'req-1e82be15-1cde-40e2-9f32-1985ebd421ba'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.834585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65e6d93672b46e19205544946e4f685', 'request_id': 'req-1e82be15-1cde-40e2-9f32-1985ebd421ba'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.835555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.836010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.836355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.839111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 994/2020] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:56:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:44.848285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] The token cache is being invalidate because project b65e6d93672b46e19205544946e4f685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:44.848573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.849180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e82be15-1cde-40e2-9f32-1985ebd421ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:44.850728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1027/2021] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:44 2022] DELETE /identity/v3/projects/b65e6d93672b46e19205544946e4f685 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:44.854849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:44.856734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.870424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db37-a404-4df7-8e9b-5ed02b0836d4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.870714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db37-a404-4df7-8e9b-5ed02b0836d4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.870989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db37-a404-4df7-8e9b-5ed02b0836d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.892846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-98c0db37-a404-4df7-8e9b-5ed02b0836d4 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:44.894063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-98c0db37-a404-4df7-8e9b-5ed02b0836d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:44.950692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a215b-c448-4a03-b3b2-486860808564 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:44.955377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:44.956033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:44.956417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:44.956751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] PATH_INFO: `/v3/users/0a81e13bded94338a9df6252230089da` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:44.957918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a81e13bded94338a9df6252230089da)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:44.959987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:44.985568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1028/2022] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:45.001217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37dd5b-d76b-429a-b887-9fa1c15517e4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.001514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37dd5b-d76b-429a-b887-9fa1c15517e4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.001766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37dd5b-d76b-429a-b887-9fa1c15517e4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.010326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1029/2023] 10.222.0.17 () {54 vars in 961 bytes} [Tue Aug 30 00:56:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:56:45.011693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.011973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.012372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.015938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a81e13bded94338a9df6252230089da'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.029212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a81e13bded94338a9df6252230089da'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.034835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] MFA Rules not processed for user `b22b14f894354d4197834889e75489a5`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:45.036333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.041745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a81e13bded94338a9df6252230089da'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.042871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a81e13bded94338a9df6252230089da'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.045633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1030/2024] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Aug 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:45.053593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:45.057029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a81e13bded94338a9df6252230089da'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.064174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 995/2025] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:44 2022] DELETE /identity/v3/users/0a81e13bded94338a9df6252230089da => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:45.064444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ea166-08d5-4f63-b715-9af25f52ae27 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:45.144485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:45.145244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.145648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.145990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.149048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.168006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:45.169634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:45.181039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1031/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:45.563543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:45.565196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.609132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:45.610176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:45.612035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.672653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2830b-b187-4a3d-9c03-842073576c67 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:45.677895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:45.678960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.679380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.679648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] PATH_INFO: `/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.680941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2f29a58e5d47a68b3492e30cde74b6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:45.683109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:45.735333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c657e98-00ca-4100-8455-2c27b6268731 service placement] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:45.739238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:45.739598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2f29a58e5d47a68b3492e30cde74b6', 'request_id': 'req-ba5e4055-0e33-47f9-a83a-67cc9725155a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.748375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2f29a58e5d47a68b3492e30cde74b6', 'request_id': 'req-ba5e4055-0e33-47f9-a83a-67cc9725155a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.766732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2f29a58e5d47a68b3492e30cde74b6', 'request_id': 'req-ba5e4055-0e33-47f9-a83a-67cc9725155a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.778552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] The token cache is being invalidate because project ba2f29a58e5d47a68b3492e30cde74b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:45.778903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.779594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ba5e4055-0e33-47f9-a83a-67cc9725155a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.781220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 996/2027] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:45 2022] DELETE /identity/v3/projects/ba2f29a58e5d47a68b3492e30cde74b6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:45.788426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785129d0-c3b9-4138-b286-1b1626004156 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.788779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785129d0-c3b9-4138-b286-1b1626004156 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.789129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785129d0-c3b9-4138-b286-1b1626004156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.807999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:45.808672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.809007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.809291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] PATH_INFO: `/v3/users/acedd2df5e604530a5cdff775054cf7e` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.810650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acedd2df5e604530a5cdff775054cf7e)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:45.811043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-785129d0-c3b9-4138-b286-1b1626004156 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:45.812265 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-785129d0-c3b9-4138-b286-1b1626004156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.812870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:45.821067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 997/2028] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:45.829074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.829360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.829629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.850897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:45.852126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.860339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 998/2029] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:45.865686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acedd2df5e604530a5cdff775054cf7e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.865916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:45.873122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acedd2df5e604530a5cdff775054cf7e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.873951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acedd2df5e604530a5cdff775054cf7e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.875144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf5b60-5d46-4955-adba-2fb500f11c31 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:45.879941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:45.880656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.880940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.881242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.881996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:45.882302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:45.884023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:45.886702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acedd2df5e604530a5cdff775054cf7e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.895693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acedd2df5e604530a5cdff775054cf7e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:45.895893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 999/2030] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:56:45.900479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:45.902393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:45.905106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1032/2031] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:45 2022] DELETE /identity/v3/users/acedd2df5e604530a5cdff775054cf7e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:45.915543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec27188-a837-40e4-ab8c-3510093b9778 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:45.981899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:45.982531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:45.982815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:45.983129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:45.983810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:45.985756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.017854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1000/2032] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:45 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:46.022912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.033956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73e8de-8245-4548-9d06-f8a5be45ba67 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.105415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.108169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.108610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.109059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.110323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.112568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.147053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1033/2033] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:46 2022] POST /identity/v3/users => generated 285 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:46.149173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.163345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.165063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ce25-d199-426a-ba52-b5feb6603814 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.166500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:46.171625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.172048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.172396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.172733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.173477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.175626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.184850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4495ea1-4139-4656-aaf6-e676857c5716 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.189960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1001/2034] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:46.196108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.197403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.197991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.198620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] PATH_INFO: `/v3/projects/1f234be758ac476b82e14afbd62ba0e5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.200979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f234be758ac476b82e14afbd62ba0e5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.201792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.204369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.212812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:46.220408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f1dd-dfb6-4fb3-bd66-a5a5f2cff542 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.224766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:46.226838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.227798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.228246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.229153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.229993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.231167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.243056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1002/2035] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:46.246707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.257081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f13b1-beb7-488d-8f1a-ac6dc3f810b1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.277435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f234be758ac476b82e14afbd62ba0e5', 'request_id': 'req-724875ba-18c7-436d-b30c-f78ea55daf07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.286259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f234be758ac476b82e14afbd62ba0e5', 'request_id': 'req-724875ba-18c7-436d-b30c-f78ea55daf07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.294719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f234be758ac476b82e14afbd62ba0e5', 'request_id': 'req-724875ba-18c7-436d-b30c-f78ea55daf07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.314747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] The token cache is being invalidate because project 1f234be758ac476b82e14afbd62ba0e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:46.315328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.316350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.318374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1034/2036] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:46 2022] DELETE /identity/v3/projects/1f234be758ac476b82e14afbd62ba0e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:46.322852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.324026 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.324438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:46.324719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.324996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.325266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] PATH_INFO: `/v3/projects/cf0ee3bd565e4884bd8d549b9b0f0bf8/users/7e7a14e6fa954ab8bec13464ed8beed1/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.338370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724875ba-18c7-436d-b30c-f78ea55daf07 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.341819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0ee3bd565e4884bd8d549b9b0f0bf8, user_id=7e7a14e6fa954ab8bec13464ed8beed1, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.343047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.343695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.343971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.344242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] PATH_INFO: `/v3/users/d670cf378fdf4da69cb5ebedd1fa5473` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.344531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee56ca-3a5c-4d10-9ca8-78111e509007 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.345380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d670cf378fdf4da69cb5ebedd1fa5473)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.347668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.358300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1003/2037] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:46 2022] PUT /identity/v3/projects/cf0ee3bd565e4884bd8d549b9b0f0bf8/users/7e7a14e6fa954ab8bec13464ed8beed1/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:46.369155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.369459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.369734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.398844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] MFA Rules not processed for user `7e7a14e6fa954ab8bec13464ed8beed1`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:46.399997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:46.405655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd670cf378fdf4da69cb5ebedd1fa5473'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.415190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd670cf378fdf4da69cb5ebedd1fa5473'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.415912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd670cf378fdf4da69cb5ebedd1fa5473'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.428491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd670cf378fdf4da69cb5ebedd1fa5473'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.437054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd670cf378fdf4da69cb5ebedd1fa5473'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:46.443472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1035/2038] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:46 2022] DELETE /identity/v3/users/d670cf378fdf4da69cb5ebedd1fa5473 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:46.498462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1004/2039] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:46.509612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.511535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:46.599995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62085-d7bd-4756-9ca0-344a745a8649 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.604846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.605861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.606327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.606818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.611931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:46.676484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.678235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8ee6-68d5-496b-8d7b-fb15d0e1f04f service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:46.689209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1036/2040] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:46.708689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:46.711195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:46.812851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf81c-4e71-42b3-96f2-7f4b0a6c2949 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:46.824902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:46.908703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:46.909542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:46.909912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:46.910416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] PATH_INFO: `/v3/projects/fd4796ba69e747f580182e09bdb18294` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:46.912752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4796ba69e747f580182e09bdb18294)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:46.915266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.004340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4796ba69e747f580182e09bdb18294', 'request_id': 'req-282176a5-9ed0-454e-9669-6e5149e97bcd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:47.019028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4796ba69e747f580182e09bdb18294', 'request_id': 'req-282176a5-9ed0-454e-9669-6e5149e97bcd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:47.036927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4796ba69e747f580182e09bdb18294', 'request_id': 'req-282176a5-9ed0-454e-9669-6e5149e97bcd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:47.054719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] The token cache is being invalidate because project fd4796ba69e747f580182e09bdb18294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:47.055325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:47.056622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-282176a5-9ed0-454e-9669-6e5149e97bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:47.062537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1005/2041] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:46 2022] DELETE /identity/v3/projects/fd4796ba69e747f580182e09bdb18294 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:47.063782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.064072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.064353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.106790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:47.108063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:47.236352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1037/2042] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:47.246274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.246609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.246888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.280905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:47.282146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:47.301727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1006/2043] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:47.306258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.320641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388385a1-8c23-4962-ae62-d9dab3c8ffde None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.332144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.333129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.333567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.334414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.335759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:47.336110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.337975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.352445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1038/2044] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:47.356490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.372632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7b5e-5d7d-44f7-bfc7-cc43be2702bd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.388631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.390676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.390954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.392860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.405221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.405221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.452838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1007/2045] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:47.457060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.468403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1941-feab-4d35-9100-830b44319ca7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.542747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.543475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.543757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.544032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.547097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.547097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.585697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1039/2046] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:47.590140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.601743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44383867-c2e8-4278-9530-bc88ba52d7bc admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.614302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.614942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.615283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.615719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.617352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.619102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.634467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1008/2047] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:47.637600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.650766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3042c4-0c52-44fb-a8b4-d5591e776db1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.655837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.656544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.656866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.657220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.657954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.660114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbce8f-4d38-486a-97c5-d1d4ca95098d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.677208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1040/2048] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:47.682607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:47.694621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9da037-23c7-4154-af82-e18e456150f9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:47.700732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:47.701431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.701710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.701980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] PATH_INFO: `/v3/projects/9926ac2d6ec245bc8c66a921b656e34f/users/ed6240d9d16040be87cbad30a939311b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.723995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9926ac2d6ec245bc8c66a921b656e34f, user_id=ed6240d9d16040be87cbad30a939311b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:47.729448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:47.747454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1009/2049] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:47 2022] PUT /identity/v3/projects/9926ac2d6ec245bc8c66a921b656e34f/users/ed6240d9d16040be87cbad30a939311b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:47.758937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:47.759279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:47.759525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:47.783220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:47.794358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] MFA Rules not processed for user `ed6240d9d16040be87cbad30a939311b`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:47.796154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:47.903295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1041/2050] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:47.906645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.003286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:48.040602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb63e7-a73f-4b36-9727-ac87bc04c87f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.051635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.052458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.053119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.054114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.054920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.057187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.096885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1010/2051] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:47 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:48.100765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.122692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05c15b-eaf6-4596-a8ae-03f95d0ab502 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.260656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.261298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.261577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.261937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.262487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.267700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.324778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1042/2052] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:48 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:48.326903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.341935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886ac1f-4130-4721-a68c-daee824ba1b0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.346847 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.348212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.348212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.348212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.349219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.351246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.370511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1011/2053] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:48.371795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.384268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474abd8-8b5e-446e-8cd1-cdc74b4f2802 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.389383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.390027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.390345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.390629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.400586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.402304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e6bc0-c07a-466a-880c-cac5af69ce4e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.421988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1043/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:48.427982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.443775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569376e4-6140-469a-99b3-4bae870429d5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.450223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.451224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.451614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.451902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] PATH_INFO: `/v3/projects/6fd1f2eb8b7745048d81a3588cb0bf88/users/e3fb3f298833474cbfa716888bd01d35/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.479471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd1f2eb8b7745048d81a3588cb0bf88, user_id=e3fb3f298833474cbfa716888bd01d35, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.482782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.504870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1012/2055] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:48 2022] PUT /identity/v3/projects/6fd1f2eb8b7745048d81a3588cb0bf88/users/e3fb3f298833474cbfa716888bd01d35/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:48.518410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.519031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.519506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.579246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] MFA Rules not processed for user `e3fb3f298833474cbfa716888bd01d35`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:48.580699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:48.662345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1044/2056] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:48.667258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.741751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da624f9-dcf2-4e77-a88e-fdcb9b7d81f5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.746577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.747232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.747508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.747780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.748429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.750340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.782407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1013/2057] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:48 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:48.787021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.798786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07beeb8f-5a81-4c93-a438-a860d3ac6b36 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.804953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:48.883965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.885988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.885988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.885988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.886261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.888388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.917782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1045/2058] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:48 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:48.922202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.935227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06618e-6f0a-4147-bbc4-77af3d445f8e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.945726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.948258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.948865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.949385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:48.950208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:48.952323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:48.967166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1014/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:56:48.972514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:48.986035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0986d76-74d7-40fd-bc59-31e1b6f1528c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:48.992197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:48.992770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:48.993043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:48.993319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] PATH_INFO: `/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9/users/09a28fccd5384c8aa95432f5376332d2/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:49.016881 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8fd25223dc4f3db3fc21a270d10ed9, user_id=09a28fccd5384c8aa95432f5376332d2, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:49.019758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:49.046363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1046/2060] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:48 2022] PUT /identity/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9/users/09a28fccd5384c8aa95432f5376332d2/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:49.050973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:49.062900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148efdc-c648-4ec7-ad67-3b96ed2d27a6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:49.126212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:49.126841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:49.127162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:49.127437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:49.128167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:49.129747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650f478-cce2-4498-a9da-917d2fb098b2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:49.137920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:49.147667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1015/2061] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:49.148911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:49.160221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f00058-f5ae-4848-ab11-2775355bd528 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:49.165236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:49.165904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:49.166176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:49.166532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] PATH_INFO: `/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9/users/09a28fccd5384c8aa95432f5376332d2/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:49.168767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8fd25223dc4f3db3fc21a270d10ed9, user_id=09a28fccd5384c8aa95432f5376332d2, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:49.171559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:49.183823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1047/2062] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:49 2022] PUT /identity/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9/users/09a28fccd5384c8aa95432f5376332d2/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:49.194770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:49.195249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:49.195740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:49.228621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] MFA Rules not processed for user `09a28fccd5384c8aa95432f5376332d2`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:49.230109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:49.337223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1016/2063] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:49.351707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:49.353692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:49.422892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d3e3-2e20-4ede-8de5-0b5fd5d17b14 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:49.426883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:49.427533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:49.427807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:49.428210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:49.492136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:49.493582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:49.503063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1048/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:51.164593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:51.166178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:51.179580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5c6c6-494e-46c7-952a-1f7f8c1742cf None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:51.183625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:51.184249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:51.184529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:51.184806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:51.245328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:51.246808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:51.258853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1017/2065] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:51.872328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:51.881532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911810b6-2f53-40af-9726-6992617a20d3 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:51.886611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:51.887329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:51.887686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:51.888026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:51.898002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:51.899707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:51.911532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1049/2066] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:55.781141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:55.782900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:55.854979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebdacd-4daf-4663-aa45-788e702bc22f service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:55.860075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:55.860404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:55.860679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:55.860948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] PATH_INFO: `/v3/users/e0e5c1345e4049f0bf5fc17ef71b7133` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:55.862059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e5c1345e4049f0bf5fc17ef71b7133)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:55.864023 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:55.896341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:55.917303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e5c1345e4049f0bf5fc17ef71b7133'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:55.924339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e5c1345e4049f0bf5fc17ef71b7133'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:55.934525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e5c1345e4049f0bf5fc17ef71b7133'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:55.935235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e5c1345e4049f0bf5fc17ef71b7133'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:55.949634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e5c1345e4049f0bf5fc17ef71b7133'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:55.955809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1018/2067] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:55 2022] DELETE /identity/v3/users/e0e5c1345e4049f0bf5fc17ef71b7133 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:56.136847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.143278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:56.157021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9ba58-8d5b-419b-b715-ca44e7019484 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.251424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.253612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.257633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.258371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.258855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.259355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] PATH_INFO: `/v3/users/ed6240d9d16040be87cbad30a939311b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.262885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6240d9d16040be87cbad30a939311b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.265241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.273011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c7f61-6b97-4585-bbb4-1438a01033db admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.278147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.278799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.279105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.279378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] PATH_INFO: `/v3/projects/1ab0b517d41b4b83af9af6f1fcce13d7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.280832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab0b517d41b4b83af9af6f1fcce13d7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.282765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.339192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6240d9d16040be87cbad30a939311b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.347684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab0b517d41b4b83af9af6f1fcce13d7', 'request_id': 'req-d663f9eb-a030-456f-b00f-3b632fab9801'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.348243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6240d9d16040be87cbad30a939311b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.348953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6240d9d16040be87cbad30a939311b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.353554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab0b517d41b4b83af9af6f1fcce13d7', 'request_id': 'req-d663f9eb-a030-456f-b00f-3b632fab9801'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.362348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6240d9d16040be87cbad30a939311b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.363684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab0b517d41b4b83af9af6f1fcce13d7', 'request_id': 'req-d663f9eb-a030-456f-b00f-3b632fab9801'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.372633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-7dfe99b4-4765-4227-a986-7ad98be8cefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6240d9d16040be87cbad30a939311b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.379829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] The token cache is being invalidate because project 1ab0b517d41b4b83af9af6f1fcce13d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:56.380118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.380474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1050/2068] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:56 2022] DELETE /identity/v3/users/ed6240d9d16040be87cbad30a939311b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:56.380928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d663f9eb-a030-456f-b00f-3b632fab9801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:56.383527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1019/2069] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:56 2022] DELETE /identity/v3/projects/1ab0b517d41b4b83af9af6f1fcce13d7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:56.388205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.388359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791940-0ece-48df-8aca-862d5502bdeb None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.388493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.388587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791940-0ece-48df-8aca-862d5502bdeb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.388834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.388945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791940-0ece-48df-8aca-862d5502bdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.415424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b6791940-0ece-48df-8aca-862d5502bdeb None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:56.415424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b6791940-0ece-48df-8aca-862d5502bdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.423766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:56.424978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.485725 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1020/2070] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:56.494112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.494400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.494679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.505861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1051/2071] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:56.512243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.513620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.518046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:56.519262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.527614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1021/2072] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:56.531690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.540934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3402-55e2-4c7f-8341-2d1752b2d531 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.541465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:56.551808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.552442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.552720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.552990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.553718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:56.554083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.555892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.569032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1022/2073] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:56:56.573737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.585728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000a90-2076-4a2d-9d48-00543569fb5f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.589000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47cfb3-4f0a-454c-a834-6817c5809d4f None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.590882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.591695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.591983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.592258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.593014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.593267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.593960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.594301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.594634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.595092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.600133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:56.604271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.605730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.627473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1052/2074] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:56.634417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1023/2075] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:56.638510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.649458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91c4b-f491-4067-8d38-5391d775a2ee service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.726480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.727384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.727685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.727940 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.728610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.730672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.760049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1053/2076] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:56.764632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.776231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1124cb-6832-4db1-832c-e0a053620c3b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.783199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.783199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.783199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.783199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.784043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.790466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.802068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1024/2077] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:56.807297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.820620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7083eb-7a49-40b4-ac0f-06f76de8510a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.825352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.825967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.826242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.826509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.827190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.828800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd845106-a842-4f1e-a3e1-e026f61bd4d7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.847380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1054/2078] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:56.848530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.860237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157f228-0709-40d8-81e7-eca29d415ab6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:56.865598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:56.866843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.867172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.867448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] PATH_INFO: `/v3/projects/d30918df8714442c882d7b64224161b2/users/298651ca2bb24a2e967fac6a9af3b02c/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.887527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30918df8714442c882d7b64224161b2, user_id=298651ca2bb24a2e967fac6a9af3b02c, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:56.890911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:56.906054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1025/2079] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:56 2022] PUT /identity/v3/projects/d30918df8714442c882d7b64224161b2/users/298651ca2bb24a2e967fac6a9af3b02c/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:56.916731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:56.917065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:56.917435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:56.963892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] MFA Rules not processed for user `298651ca2bb24a2e967fac6a9af3b02c`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:56.965171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:56.992978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:56.995588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.062845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1055/2080] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:57.067850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.082201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ecf50-20d9-4a2d-a96b-a6ff06849ea3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.100353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.101126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.101407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.101680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] PATH_INFO: `/v3/projects/9926ac2d6ec245bc8c66a921b656e34f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.103448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9926ac2d6ec245bc8c66a921b656e34f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.103561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:57.105318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.139582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c99f68-e9ef-4b58-b5e0-3d05fcb20f84 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.144374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.144996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.145269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.145538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.146176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.148435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.182048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9926ac2d6ec245bc8c66a921b656e34f', 'request_id': 'req-bcca54d0-5b25-44f2-a592-67f80cf8f654'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.186778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1056/2081] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:57 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:56:57.188178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.192836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9926ac2d6ec245bc8c66a921b656e34f', 'request_id': 'req-bcca54d0-5b25-44f2-a592-67f80cf8f654'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.202527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9926ac2d6ec245bc8c66a921b656e34f', 'request_id': 'req-bcca54d0-5b25-44f2-a592-67f80cf8f654'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.214742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] The token cache is being invalidate because project 9926ac2d6ec245bc8c66a921b656e34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:57.215051 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.215765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.217431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1026/2082] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:56 2022] DELETE /identity/v3/projects/9926ac2d6ec245bc8c66a921b656e34f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:57.221374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.222784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.276010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e17e7d-f11c-4e09-b5ee-ccc95e936618 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.280563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:57.285518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.286134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.286602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.287097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.287918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.291065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.313379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca54d0-5b25-44f2-a592-67f80cf8f654 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.319821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.320672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.320899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.321148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] PATH_INFO: `/v3/users/e3fb3f298833474cbfa716888bd01d35` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.322515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3fb3f298833474cbfa716888bd01d35)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.324706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.337408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1057/2083] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:57 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:57.341418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.344185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.359407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e56b-1321-4734-ba7e-4bc2c99c05fa admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.364487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.365113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.365436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.366027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.366908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.368828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.384031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1058/2084] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:57.385190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.395475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3fb3f298833474cbfa716888bd01d35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.400561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb731b-f5b3-4bc0-bc96-8a2f1c85b49e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.403305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3fb3f298833474cbfa716888bd01d35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.405467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.406144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.406432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.406709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] PATH_INFO: `/v3/projects/eed42af25d9e4747ae36b044f7107def/users/fa1b1d3282be40dc9b3ce1078042a26c/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.412110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3fb3f298833474cbfa716888bd01d35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.412770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3fb3f298833474cbfa716888bd01d35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.426704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3fb3f298833474cbfa716888bd01d35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.427509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed42af25d9e4747ae36b044f7107def, user_id=fa1b1d3282be40dc9b3ce1078042a26c, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.431736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.445856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1027/2085] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:57 2022] DELETE /identity/v3/users/e3fb3f298833474cbfa716888bd01d35 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:57.460199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1059/2086] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:57 2022] PUT /identity/v3/projects/eed42af25d9e4747ae36b044f7107def/users/fa1b1d3282be40dc9b3ce1078042a26c/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:57.464922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.466472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.537815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4212ce-027e-49f8-b099-9b18331556d7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.606227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.606227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.606227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.606227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.606547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.607148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cedcb-e292-44ac-9a17-a7cfaa279c09 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.619337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1028/2087] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:57.624009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.637255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760284-3ce6-4921-aa71-63a1bfc269b5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.641856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.642521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.642805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.643109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] PATH_INFO: `/v3/projects/eed42af25d9e4747ae36b044f7107def/users/fa1b1d3282be40dc9b3ce1078042a26c/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.645011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed42af25d9e4747ae36b044f7107def, user_id=fa1b1d3282be40dc9b3ce1078042a26c, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.647876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.658464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1060/2088] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:57 2022] PUT /identity/v3/projects/eed42af25d9e4747ae36b044f7107def/users/fa1b1d3282be40dc9b3ce1078042a26c/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:57.670223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.670522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.670798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.678187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:57.706434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] MFA Rules not processed for user `fa1b1d3282be40dc9b3ce1078042a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:57.707760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.746535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.748505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.801224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1029/2089] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:57.813336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.815126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.828797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f820c44-5ad5-4102-8623-8c2e966e1481 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.834000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.834793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.835198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.835480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] PATH_INFO: `/v3/projects/6fd1f2eb8b7745048d81a3588cb0bf88` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.836606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd1f2eb8b7745048d81a3588cb0bf88)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:57.838533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:57.840679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:57.887558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895990-a68a-4df1-91de-aa8e1a77a216 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:57.893782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd1f2eb8b7745048d81a3588cb0bf88', 'request_id': 'req-2e3d3274-f6b0-4518-b8fa-954a3f194df2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.901073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd1f2eb8b7745048d81a3588cb0bf88', 'request_id': 'req-2e3d3274-f6b0-4518-b8fa-954a3f194df2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.907711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd1f2eb8b7745048d81a3588cb0bf88', 'request_id': 'req-2e3d3274-f6b0-4518-b8fa-954a3f194df2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.923658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] The token cache is being invalidate because project 6fd1f2eb8b7745048d81a3588cb0bf88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:57.924125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.924818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:57.926564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1061/2090] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:57 2022] DELETE /identity/v3/projects/6fd1f2eb8b7745048d81a3588cb0bf88 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:57.930931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:57.932452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:57.952267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:57.952926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:57.953240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:57.953576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:57.955900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.009722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3274-f6b0-4518-b8fa-954a3f194df2 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.014813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:58.015485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.015761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.016029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] PATH_INFO: `/v3/users/09a28fccd5384c8aa95432f5376332d2` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.017261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a28fccd5384c8aa95432f5376332d2)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.019146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.033104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:58.035720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.037721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.047998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1030/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:56:58.090010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a28fccd5384c8aa95432f5376332d2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.099273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a28fccd5384c8aa95432f5376332d2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.100445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a28fccd5384c8aa95432f5376332d2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.116104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a28fccd5384c8aa95432f5376332d2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.128600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a28fccd5384c8aa95432f5376332d2'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.133920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:58.135948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.139330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1062/2092] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:56:57 2022] DELETE /identity/v3/users/09a28fccd5384c8aa95432f5376332d2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:56:58.150816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acafbcf-1469-4828-93ce-a8a65df6dd50 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.210647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:58.211341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.211622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.211891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.213827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.283601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.285109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e633dc1-5a77-4679-aabb-4cbb51fcfd2a service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.295444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1031/2093] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:56:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:56:58.502886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:58.505162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.516300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:58.589796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a266c-9b80-42ed-887f-838c84dbf792 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.594613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:58.595296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.595577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.595845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] PATH_INFO: `/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.597152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8fd25223dc4f3db3fc21a270d10ed9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.599102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.661023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8fd25223dc4f3db3fc21a270d10ed9', 'request_id': 'req-aec7a836-938b-419c-8e3f-5f7ca5eae26a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.668036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8fd25223dc4f3db3fc21a270d10ed9', 'request_id': 'req-aec7a836-938b-419c-8e3f-5f7ca5eae26a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.674871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8fd25223dc4f3db3fc21a270d10ed9', 'request_id': 'req-aec7a836-938b-419c-8e3f-5f7ca5eae26a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.689619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] The token cache is being invalidate because project 3b8fd25223dc4f3db3fc21a270d10ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:56:58.689908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.690540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-aec7a836-938b-419c-8e3f-5f7ca5eae26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:56:58.692160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1063/2094] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:56:58 2022] DELETE /identity/v3/projects/3b8fd25223dc4f3db3fc21a270d10ed9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:58.698636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.699074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.699439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.726922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:58.728245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.801039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1032/2095] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:58.809458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.809764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.810034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.832589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:58.833819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:58.842657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1064/2096] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:58.847176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:58.858962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780eaa5-3c49-4647-91ba-65f880f6588c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.865025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:58.865668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.866175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.866303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.866897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:56:58.867573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.870272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.886172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1033/2097] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:56:58.890892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:58.901045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393c970-9713-4a3d-97c4-895159e57e8e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.907053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:58.907690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:58.907974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:58.908240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:58.908876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:58.910774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:58.945797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1065/2098] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:58 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:58.950078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:58.960574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc9d3-71b1-4354-b79d-3cae1c680612 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:58.998458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:59.025899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.026506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.026792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.027093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.027739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.029664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.057500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1034/2099] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:58 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:59.061542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.075032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f0cb1-601f-4b63-b93a-952758d2dc06 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.080139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.080770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.081043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.081506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.082160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.083829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.097058 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1066/2100] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:59.100324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.111119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb3cec-0d49-471d-8abd-ee7a90246107 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.116090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.116699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.116973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.117352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.118046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.119748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba9a83-85f9-4630-bbf1-3d7c621a63ef admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.136247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1035/2101] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:59.140801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.151796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a5ca-ed7f-4423-8438-18ddef65d21e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.157790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.158442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.158718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.158991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] PATH_INFO: `/v3/projects/454302103e1a43ec8607b7d48ea4a043/users/ab5cdb615c034faa9566d9bd42a30349/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.181986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454302103e1a43ec8607b7d48ea4a043, user_id=ab5cdb615c034faa9566d9bd42a30349, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.185296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.203578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1067/2102] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:59 2022] PUT /identity/v3/projects/454302103e1a43ec8607b7d48ea4a043/users/ab5cdb615c034faa9566d9bd42a30349/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:59.216937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.217224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.217524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.249147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] MFA Rules not processed for user `ab5cdb615c034faa9566d9bd42a30349`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:59.250394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:59.324650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1036/2103] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:56:59.329181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.374985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:59.399835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad8932-15f0-434f-ba9a-f4d70f7edf41 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.405280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.405909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.406350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.406627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.407312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.411935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.444473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1068/2104] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:59.445681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.456541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4735a1d-68d3-4144-9b38-2f1818fd6009 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.523694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.524753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.525329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.525644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.526658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.528878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.564142 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1037/2105] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:59.567569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.580582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1c5f7-d433-4140-b4e8-610c35268ba7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.587154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.587892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.588175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.588419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.589103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.590777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.602571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1069/2106] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:59.607359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.622232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8114b2-bb00-4fb6-be0a-8f91dc03b1bb admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.627088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.627706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.627985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.628257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.628898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.630513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996471f-1910-45a8-8120-9bbe629f9057 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.644757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1038/2107] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:56:59.645707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.656884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2e5d-01ef-4d0b-9abc-92615b4c01ce admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.662796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.663494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.663771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.664047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] PATH_INFO: `/v3/projects/7823add607174a588bcbe0b74a7279c2/users/e24b3e932bda42c2abcd3a311b47f117/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.680185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7823add607174a588bcbe0b74a7279c2, user_id=e24b3e932bda42c2abcd3a311b47f117, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.682892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.705357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1070/2108] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:56:59 2022] PUT /identity/v3/projects/7823add607174a588bcbe0b74a7279c2/users/e24b3e932bda42c2abcd3a311b47f117/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:56:59.716134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.716320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.716575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.752611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] MFA Rules not processed for user `e24b3e932bda42c2abcd3a311b47f117`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:56:59.753858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:56:59.763972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:56:59.834740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1039/2109] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:56:59.842132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.926585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f37-8520-47d2-9929-ed4c33ea1aaf admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:56:59.934159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:56:59.934806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:56:59.935267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:56:59.935676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:56:59.936414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:56:59.938801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:56:59.979938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1071/2110] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:56:59.984098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:56:59.996830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c174-d44f-4050-a3e5-2b400e7e0d93 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.073924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.074545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.074827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.075193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.075770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.077682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.099803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1040/2111] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:56:59 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:00.103976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:00.115421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2eda5-b8e3-4e7d-9914-184308222c6e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.121986 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.122721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.123069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.123410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.124133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.125833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.139131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1072/2112] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:00.144036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:00.155589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304dd28-3687-4f03-bddd-27b9823e60c0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.159649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.160333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.160648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.160952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] PATH_INFO: `/v3/projects/b54db40155734ace9b8e2df87d14ad42/users/ded3a5fda2214932818afd876a4740ef/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.182342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54db40155734ace9b8e2df87d14ad42, user_id=ded3a5fda2214932818afd876a4740ef, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.191027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.212760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1041/2113] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:00 2022] PUT /identity/v3/projects/b54db40155734ace9b8e2df87d14ad42/users/ded3a5fda2214932818afd876a4740ef/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:00.218018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:00.231389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788a5c-596e-4d01-a7b6-d699f4e2b6f5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.296771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.297826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.298276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.298709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.300021 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.302939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28543f-696d-476a-ac52-f0a281a5ecc2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.307570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:00.327868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1073/2114] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:00.331894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:00.345238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f210f-5bc0-41d1-b0d1-8a2748dccf01 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.350588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.351518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.351776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.352049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] PATH_INFO: `/v3/projects/b54db40155734ace9b8e2df87d14ad42/users/ded3a5fda2214932818afd876a4740ef/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.354237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54db40155734ace9b8e2df87d14ad42, user_id=ded3a5fda2214932818afd876a4740ef, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.356944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.375433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1042/2115] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:00 2022] PUT /identity/v3/projects/b54db40155734ace9b8e2df87d14ad42/users/ded3a5fda2214932818afd876a4740ef/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:00.391254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.391542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.391886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.425552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] MFA Rules not processed for user `ded3a5fda2214932818afd876a4740ef`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:00.426887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:00.530584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1074/2116] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:00.539261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:00.541141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:00.638419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7798d-f801-45f6-886c-b819a9f56569 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:00.645009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:00.645624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:00.645905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:00.646172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:00.672054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:00.730315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:00.731735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:00.741693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1043/2117] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:02.379993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:02.391351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6a0b-8ab7-49cf-ba7d-2982219912b3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:02.396391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:02.397273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:02.397608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:02.397877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:02.459212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:02.460670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65891090-46d0-41be-82cf-68e118283e52 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:02.469608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1075/2118] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:04.035323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:04.036754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:04.048576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3efae-1c98-440c-93e3-671b85a69995 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:04.052539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:04.053143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:04.053419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:04.053686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:04.063498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:04.064957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:04.075843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1044/2119] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:08.968192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693df6d2-e0db-4dfe-906b-aa657e90c185 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:08.968874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693df6d2-e0db-4dfe-906b-aa657e90c185 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:08.969095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693df6d2-e0db-4dfe-906b-aa657e90c185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:08.990932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-693df6d2-e0db-4dfe-906b-aa657e90c185 None None] MFA Rules not processed for user `309da81b8e13455a836e0c8435ed6b06`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:08.992471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-693df6d2-e0db-4dfe-906b-aa657e90c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:09.082391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1076/2120] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:12.505080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:12.577730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040be2-1bb9-4cfc-aadd-43950a45bd1e service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:12.582423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:12.583097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:12.583376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:12.583646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] PATH_INFO: `/v3/users/ab5cdb615c034faa9566d9bd42a30349` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:12.584780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5cdb615c034faa9566d9bd42a30349)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:12.586724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:12.643483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5cdb615c034faa9566d9bd42a30349'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:12.650320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5cdb615c034faa9566d9bd42a30349'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:12.660070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5cdb615c034faa9566d9bd42a30349'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:12.660930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5cdb615c034faa9566d9bd42a30349'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:12.674917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5cdb615c034faa9566d9bd42a30349'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:12.681092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1045/2121] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:12 2022] DELETE /identity/v3/users/ab5cdb615c034faa9566d9bd42a30349 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:12.685838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:12.686319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:12.686500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:12.706366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:12.707641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:12.726661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:12.772561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1077/2122] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:12.783674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:12.785275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:12.820508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:12.858705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b5f9-6139-4eb3-9643-7d61721c45cc admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:12.862376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:12.862986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:12.863307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:12.863583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:12.873214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:12.874651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:12.885009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1046/2123] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:13.343253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:13.345180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:13.373918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d680860-56b7-4420-8eaa-24af63c42e18 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:13.381710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:13.384190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:13.398312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:13.398801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] PATH_INFO: `/v3/projects/454302103e1a43ec8607b7d48ea4a043` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:13.406863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=454302103e1a43ec8607b7d48ea4a043)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:13.413762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:13.524657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454302103e1a43ec8607b7d48ea4a043', 'request_id': 'req-5dfcc2d6-6579-4832-8a34-820f69647524'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.532234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454302103e1a43ec8607b7d48ea4a043', 'request_id': 'req-5dfcc2d6-6579-4832-8a34-820f69647524'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.538676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454302103e1a43ec8607b7d48ea4a043', 'request_id': 'req-5dfcc2d6-6579-4832-8a34-820f69647524'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.554333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] The token cache is being invalidate because project 454302103e1a43ec8607b7d48ea4a043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:13.554654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.555327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.557579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1078/2124] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:13 2022] DELETE /identity/v3/projects/454302103e1a43ec8607b7d48ea4a043 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:13.561622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:13.572151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84666bda-849f-46ee-9880-f7f962de3485 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:13.644569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:13.646134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:13.664075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:13.664614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:13.664892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:13.665385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] PATH_INFO: `/v3/users/e24b3e932bda42c2abcd3a311b47f117` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:13.667802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24b3e932bda42c2abcd3a311b47f117)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:13.670250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:13.747873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc2d6-6579-4832-8a34-820f69647524 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:13.764203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24b3e932bda42c2abcd3a311b47f117'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.768664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:13.769355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:13.769632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:13.769898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] PATH_INFO: `/v3/users/298651ca2bb24a2e967fac6a9af3b02c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:13.784270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298651ca2bb24a2e967fac6a9af3b02c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:13.784270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:13.786044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24b3e932bda42c2abcd3a311b47f117'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.800729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24b3e932bda42c2abcd3a311b47f117'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.803126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24b3e932bda42c2abcd3a311b47f117'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.829371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-43986042-a7a1-40cd-85f1-6f544d3fca77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24b3e932bda42c2abcd3a311b47f117'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.841997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1047/2125] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:13 2022] DELETE /identity/v3/users/e24b3e932bda42c2abcd3a311b47f117 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:13.910375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298651ca2bb24a2e967fac6a9af3b02c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.916879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:13.931416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298651ca2bb24a2e967fac6a9af3b02c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.932465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298651ca2bb24a2e967fac6a9af3b02c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.957674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298651ca2bb24a2e967fac6a9af3b02c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.968018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298651ca2bb24a2e967fac6a9af3b02c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:13.978627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1079/2126] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:13 2022] DELETE /identity/v3/users/298651ca2bb24a2e967fac6a9af3b02c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:13.984071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:13.984485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:13.985231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.008931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:14.010143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.058106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:14.079115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1048/2127] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:14.089173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.090672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.150483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b22a-f5f0-4ea4-b984-be71d81e1494 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.153404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.154909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.155094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.155705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.157510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.157510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.169956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.171519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.183990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd1bf4-1796-4163-99f9-37bc6b4e04e4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.185593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1080/2128] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:14.194237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.195096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.202463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.202463 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] PATH_INFO: `/v3/projects/7823add607174a588bcbe0b74a7279c2` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.206331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7823add607174a588bcbe0b74a7279c2)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.206331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.263885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7823add607174a588bcbe0b74a7279c2', 'request_id': 'req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.269319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7823add607174a588bcbe0b74a7279c2', 'request_id': 'req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.278363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7823add607174a588bcbe0b74a7279c2', 'request_id': 'req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.289809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] The token cache is being invalidate because project 7823add607174a588bcbe0b74a7279c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:14.290207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.291058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.292619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1049/2129] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:14 2022] DELETE /identity/v3/projects/7823add607174a588bcbe0b74a7279c2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:14.297286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.307441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f335-e5ca-4f37-9b50-0c2c477144cf service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.370827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.371505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.371784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.372056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] PATH_INFO: `/v3/users/ded3a5fda2214932818afd876a4740ef` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.373207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded3a5fda2214932818afd876a4740ef)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.375180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.446926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded3a5fda2214932818afd876a4740ef'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.454511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded3a5fda2214932818afd876a4740ef'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.455345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded3a5fda2214932818afd876a4740ef'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.467259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:14.469272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded3a5fda2214932818afd876a4740ef'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.480016 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded3a5fda2214932818afd876a4740ef'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.487112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1081/2130] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:14 2022] DELETE /identity/v3/users/ded3a5fda2214932818afd876a4740ef => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:14.575149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.577036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.590814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce2a6f-14a8-4c9a-ba8e-1d88cfc7904d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.658096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.658753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.659141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.659417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] PATH_INFO: `/v3/projects/d30918df8714442c882d7b64224161b2` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.660623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30918df8714442c882d7b64224161b2)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.662548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.715781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30918df8714442c882d7b64224161b2', 'request_id': 'req-0f66cac0-0546-448d-b8eb-81484b0ab660'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.721372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30918df8714442c882d7b64224161b2', 'request_id': 'req-0f66cac0-0546-448d-b8eb-81484b0ab660'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.730402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30918df8714442c882d7b64224161b2', 'request_id': 'req-0f66cac0-0546-448d-b8eb-81484b0ab660'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.738416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:14.743998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] The token cache is being invalidate because project d30918df8714442c882d7b64224161b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:14.744919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.745426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.747739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1050/2131] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:14 2022] DELETE /identity/v3/projects/d30918df8714442c882d7b64224161b2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:14.752000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.753508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.767636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032796-889b-4da7-ab99-09005d3b6867 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.788926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:14.790935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:14.842938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.843620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.843897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.844171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] PATH_INFO: `/v3/users/fa1b1d3282be40dc9b3ce1078042a26c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.845301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1b1d3282be40dc9b3ce1078042a26c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.847303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.864908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66cac0-0546-448d-b8eb-81484b0ab660 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:14.870176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:14.870810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:14.871121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:14.871401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] PATH_INFO: `/v3/projects/b54db40155734ace9b8e2df87d14ad42` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:14.872681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54db40155734ace9b8e2df87d14ad42)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:14.874577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:14.906195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1b1d3282be40dc9b3ce1078042a26c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.915607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1b1d3282be40dc9b3ce1078042a26c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.916631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1b1d3282be40dc9b3ce1078042a26c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.931754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1b1d3282be40dc9b3ce1078042a26c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.945212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54db40155734ace9b8e2df87d14ad42', 'request_id': 'req-39bf29d4-66b1-46c1-a72e-90f737f3bad2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.948496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8c5df7f3-a805-4225-b66c-6eab6d4a0f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1b1d3282be40dc9b3ce1078042a26c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.958192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54db40155734ace9b8e2df87d14ad42', 'request_id': 'req-39bf29d4-66b1-46c1-a72e-90f737f3bad2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.969997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54db40155734ace9b8e2df87d14ad42', 'request_id': 'req-39bf29d4-66b1-46c1-a72e-90f737f3bad2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.986991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1082/2132] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:14 2022] DELETE /identity/v3/users/fa1b1d3282be40dc9b3ce1078042a26c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:14.991876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] The token cache is being invalidate because project b54db40155734ace9b8e2df87d14ad42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:14.992285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.993738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-39bf29d4-66b1-46c1-a72e-90f737f3bad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:14.999497 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1051/2133] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:14 2022] DELETE /identity/v3/projects/b54db40155734ace9b8e2df87d14ad42 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:15.002907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.003350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.003874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.037259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:15.039875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.163566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1083/2134] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:15.166138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.166990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.166990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.200959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:15.202803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.217678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1052/2135] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:15.222038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.233712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc02a38-f2da-471a-ad1e-6be484cdcf0e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.238664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.239426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.239773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.240115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.240851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:15.241224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.242921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.245876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:15.261667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1084/2136] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:15.263397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.275730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ea6c5-246a-401c-9335-3e0d2bc38b27 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.280986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.281615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.281950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.282229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.282883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.285009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.374782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1053/2137] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:15.378985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.395453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e677d-be3e-4e7f-9b95-14ec4742479c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.433454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.435448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.519959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.520861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.521207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.522366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.523139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.525479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.527258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ee31d-6e97-4ebb-88f5-eec8526249a0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.534314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.534981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.535289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.535561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] PATH_INFO: `/v3/projects/eed42af25d9e4747ae36b044f7107def` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.540316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed42af25d9e4747ae36b044f7107def)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.548271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.552200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:15.562603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1085/2138] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:15.563946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.579151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b1c3-ecb8-4539-bbf8-15bfa4e9b01e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.586310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.587330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.587757 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.588343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.589379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.592711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.612913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1086/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:15.614483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.627627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db1935-e713-47e5-9b59-2caa365b1deb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.635059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed42af25d9e4747ae36b044f7107def', 'request_id': 'req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:15.643431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed42af25d9e4747ae36b044f7107def', 'request_id': 'req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:15.658974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed42af25d9e4747ae36b044f7107def', 'request_id': 'req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:15.677402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] The token cache is being invalidate because project eed42af25d9e4747ae36b044f7107def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:15.677732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:15.678638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c3c9bf61-f4c0-48c3-bb11-22c1cf6bc59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:15.681013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1054/2140] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:15 2022] DELETE /identity/v3/projects/eed42af25d9e4747ae36b044f7107def => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:15.687274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b675a-5108-4b31-886b-27ac4aed8385 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.687593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b675a-5108-4b31-886b-27ac4aed8385 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.687879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b675a-5108-4b31-886b-27ac4aed8385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.714481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.715150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-302b675a-5108-4b31-886b-27ac4aed8385 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:15.716133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.716414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.716696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-302b675a-5108-4b31-886b-27ac4aed8385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.716984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.717819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.719808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.726943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1055/2141] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:15.735363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.735656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.735915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.742740 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1087/2142] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:15.744712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.746868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.762216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:15.762424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fafe78-ce14-4c35-8ce5-7184410e5a3c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.763682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.768015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.769032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.769380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.769723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] PATH_INFO: `/v3/projects/a75a3c55fe97478785ed455b2a20c3e6/users/a9542c87fdd54d7580182667379c46e4/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.773205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1056/2143] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:15.778363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.789239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75a3c55fe97478785ed455b2a20c3e6, user_id=a9542c87fdd54d7580182667379c46e4, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.789949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d76451-6a7d-4f49-b996-acbfff821ad6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.792762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e246e83-2116-48d3-b35d-8857c1d601c9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.795626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.796415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.796757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.797079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.798124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:15.798405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.800442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.823937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1057/2145] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:57:15.823937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.824226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1088/2145] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:15 2022] PUT /identity/v3/projects/a75a3c55fe97478785ed455b2a20c3e6/users/a9542c87fdd54d7580182667379c46e4/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:15.827825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.828124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.828403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.833220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd67db-dd43-45e3-8045-13eb6558e585 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:15.873574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] MFA Rules not processed for user `a9542c87fdd54d7580182667379c46e4`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:15.874837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:15.900681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:15.901483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:15.901917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:15.902354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:15.902983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:15.904987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:15.913918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:15.938939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1058/2146] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:15.943581 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.954102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1089/2147] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:15.967498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:15.969081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:16.005387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:16.021917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627293b-c5a5-4b21-b742-adfef869055f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.028203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.028834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.029106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.029378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.030020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.031096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3872058-4584-4006-adb6-318ca5dee6dd None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.032004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.055948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1059/2148] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:15 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:16.060597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.071091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283541d-cad3-4186-9a1d-6abc451658c1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.076554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.077201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.077465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.077739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.078385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.080165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.084230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.084861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.085140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.085410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.092030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1060/2149] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:16.096210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.106144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df784a0-c222-45e3-95b8-28efbaf60708 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.110916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.111523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.111796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.112099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.112687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.114297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.124276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1061/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:16.128406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.138999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bb88b-e70e-44cc-8ab8-d828a869d1d5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.144040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.144677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.144951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.145259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] PATH_INFO: `/v3/projects/202ac50b78614187afba426663ba3de8/users/320760b02db243809708d3f371859919/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.147093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.148675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bbbb6-2d24-4d8b-895d-db283598cc96 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.158319 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1090/2151] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:57:16.160903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202ac50b78614187afba426663ba3de8, user_id=320760b02db243809708d3f371859919, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.165605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.186223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1062/2152] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:16 2022] PUT /identity/v3/projects/202ac50b78614187afba426663ba3de8/users/320760b02db243809708d3f371859919/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:16.197203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.197203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.197468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.225651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] MFA Rules not processed for user `320760b02db243809708d3f371859919`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:16.226867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:16.279602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:16.324130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1091/2153] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:16.335775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.397163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed075d0-92d6-4ad6-8f3e-2546f2d27153 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.398328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.398939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.399242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.399815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.409845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.411475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c2cb8-7cd5-47e2-8e17-06cabba73fbf service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.421632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1063/2154] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:16.440953 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.514006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e331e0a-b16f-4b0a-ac87-a4068dbe555c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.519320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.519954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.520229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.520495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] PATH_INFO: `/v3/users/320760b02db243809708d3f371859919` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.521660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=320760b02db243809708d3f371859919)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.523633 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.578371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320760b02db243809708d3f371859919'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:16.587682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320760b02db243809708d3f371859919'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:16.588421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320760b02db243809708d3f371859919'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:16.601650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320760b02db243809708d3f371859919'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:16.610884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320760b02db243809708d3f371859919'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:16.624255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1092/2155] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:16 2022] DELETE /identity/v3/users/320760b02db243809708d3f371859919 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:16.631877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.632184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.632666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.654607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:16.655915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:16.733778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1064/2156] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:16.744201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:16.745879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:16.807651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d566a4-ebfb-4de8-8777-63d3ac87fc3c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:16.811719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:16.812375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:16.812654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:16.812917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:16.822724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:16.824219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ad402-e8d2-46f4-849f-5cc70a2ef0a3 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:16.828428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:16.839225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1093/2157] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:17.208107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.209534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:17.222320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd69fb-9451-477c-9155-d703e732c661 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.227817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.228466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.228898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.229211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] PATH_INFO: `/v3/projects/202ac50b78614187afba426663ba3de8` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.230494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=202ac50b78614187afba426663ba3de8)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.232478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.271219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:17.283873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202ac50b78614187afba426663ba3de8', 'request_id': 'req-006a4ddf-d690-4487-b824-e0653c708fd8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:17.289246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202ac50b78614187afba426663ba3de8', 'request_id': 'req-006a4ddf-d690-4487-b824-e0653c708fd8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:17.298540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202ac50b78614187afba426663ba3de8', 'request_id': 'req-006a4ddf-d690-4487-b824-e0653c708fd8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:17.310442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] The token cache is being invalidate because project 202ac50b78614187afba426663ba3de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:17.310731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:17.311382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-006a4ddf-d690-4487-b824-e0653c708fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:17.312882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1065/2158] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:17 2022] DELETE /identity/v3/projects/202ac50b78614187afba426663ba3de8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:17.318292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c189-502c-44be-8540-cae13b19a79e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.318580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c189-502c-44be-8540-cae13b19a79e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.318844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c189-502c-44be-8540-cae13b19a79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.339938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5177c189-502c-44be-8540-cae13b19a79e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:17.341146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5177c189-502c-44be-8540-cae13b19a79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:17.414242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1094/2159] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:17.423919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.424212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.424512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.447356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:17.448740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:17.460363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1066/2160] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:17.464849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177c189-502c-44be-8540-cae13b19a79e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.475081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177c189-502c-44be-8540-cae13b19a79e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.479855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.480504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.480782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.481078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.481731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:17.482029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.483683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.497192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1095/2161] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:17.501117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.511789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e201f97-f2e0-424c-b11d-ea4329447d8b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.516172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.516785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.517064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.517331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.517963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.520163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.555074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1067/2162] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:17 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:17.556311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.566341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210d737-e137-4b97-a2ff-4396e7a92c32 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.623759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.624373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.626052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.626329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.626974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.628975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.658202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1096/2163] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:17 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:17.663286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.676043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2688f-f326-4952-a0f9-36c54ad1dfea admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.683884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.684538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.684857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.685169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.685854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.687948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.701509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1068/2164] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:17.705766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.717029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f07824-2e06-4552-828d-22f7418b1b15 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.722031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.722636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.722920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.723232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.723882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.725480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0f36e-f30b-4504-b189-56b720cc4ec8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.736783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1097/2165] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:17.741138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.752546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a22dfe-2084-4071-a1d1-d6074baa6de9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:17.757385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:17.758041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.758319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.758577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] PATH_INFO: `/v3/projects/31a58d46765b42ef8fb5bf489971f58f/users/66b718c35c01416d87bea1876116e56e/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.773050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a58d46765b42ef8fb5bf489971f58f, user_id=66b718c35c01416d87bea1876116e56e, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:17.775894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:17.791371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1069/2166] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:17 2022] PUT /identity/v3/projects/31a58d46765b42ef8fb5bf489971f58f/users/66b718c35c01416d87bea1876116e56e/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:17.802630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:17.803082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:17.803369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:17.833720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] MFA Rules not processed for user `66b718c35c01416d87bea1876116e56e`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:17.834931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:17.905189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:17.921447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1098/2167] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:17.933537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:17.936351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:18.004362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461485d-10d3-4466-8ab2-589ff59c105d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:18.008633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:18.009258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:18.009523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:18.009794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:18.019839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:18.021285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:18.031377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1070/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:26.488637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:26.490220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:26.556189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:26.561155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:26.599223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeebae6-2b88-4126-b79d-a67f316dd983 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:26.607184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:26.609001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:26.610128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:26.610408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] PATH_INFO: `/v3/users/7e7a14e6fa954ab8bec13464ed8beed1` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:26.612083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7a14e6fa954ab8bec13464ed8beed1)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:26.615420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:26.659350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ea0-38d5-4f01-8d8a-3011f0e2e2d2 service glance] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:26.669808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:26.670502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:26.671318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:26.671593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] PATH_INFO: `/v3/users/a9542c87fdd54d7580182667379c46e4` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:26.674320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9542c87fdd54d7580182667379c46e4)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:26.676337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:26.681350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:26.692744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7a14e6fa954ab8bec13464ed8beed1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.706028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7a14e6fa954ab8bec13464ed8beed1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.706790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7a14e6fa954ab8bec13464ed8beed1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.721822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7a14e6fa954ab8bec13464ed8beed1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.746439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-31200e9e-6b3a-4bc9-b4fc-10ac2962209c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7a14e6fa954ab8bec13464ed8beed1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.753563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1099/2169] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:26 2022] DELETE /identity/v3/users/7e7a14e6fa954ab8bec13464ed8beed1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:26.758065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:26.758350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:26.758619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:26.781422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9542c87fdd54d7580182667379c46e4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.786320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:26.787836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:26.791744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9542c87fdd54d7580182667379c46e4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.807731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9542c87fdd54d7580182667379c46e4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.810358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9542c87fdd54d7580182667379c46e4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.824198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84bc3bea-197e-4d51-8fd5-e17768343e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9542c87fdd54d7580182667379c46e4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:26.830705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1071/2170] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:26 2022] DELETE /identity/v3/users/a9542c87fdd54d7580182667379c46e4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:26.835344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:26.835635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:26.835967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:26.858271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:26.859531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:26.872184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1100/2171] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:26.883117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:26.885954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:26.920316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:26.942249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1072/2172] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:26.962412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:26.966025 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.031105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3a5b7-1501-4e7f-b593-a64a11ee426a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.055978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1ea79-ef41-4918-a7ed-bc809754cbac None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.066073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.067126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.067422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.067690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.082940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.084627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.096093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.096725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.097002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.097273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.103822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1073/2173] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:27.118556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.120213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.134383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1101/2174] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:57:27.649547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.651298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.668625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.670500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.680961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ae32d-293a-40e6-9db1-c787140e4e19 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.684723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d37f7-fae1-41eb-8f12-9d59ac79c348 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.686653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.687373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.687774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.687997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] PATH_INFO: `/v3/projects/cf0ee3bd565e4884bd8d549b9b0f0bf8` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.689344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0ee3bd565e4884bd8d549b9b0f0bf8)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.690520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.691325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.691880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.692311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.692794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] PATH_INFO: `/v3/projects/a75a3c55fe97478785ed455b2a20c3e6` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.694248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75a3c55fe97478785ed455b2a20c3e6)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.696339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.715065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:27.749549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75a3c55fe97478785ed455b2a20c3e6', 'request_id': 'req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.754164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0ee3bd565e4884bd8d549b9b0f0bf8', 'request_id': 'req-a96bb89b-929b-471d-975e-f0a84da6f0a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.757559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75a3c55fe97478785ed455b2a20c3e6', 'request_id': 'req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.760662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0ee3bd565e4884bd8d549b9b0f0bf8', 'request_id': 'req-a96bb89b-929b-471d-975e-f0a84da6f0a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.765194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75a3c55fe97478785ed455b2a20c3e6', 'request_id': 'req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.771270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0ee3bd565e4884bd8d549b9b0f0bf8', 'request_id': 'req-a96bb89b-929b-471d-975e-f0a84da6f0a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.780367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] The token cache is being invalidate because project a75a3c55fe97478785ed455b2a20c3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:27.780658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.781335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-4afe0f46-df07-48e6-a2d0-c4d5dc021c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.783098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1102/2175] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:27 2022] DELETE /identity/v3/projects/a75a3c55fe97478785ed455b2a20c3e6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:27.785719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] The token cache is being invalidate because project cf0ee3bd565e4884bd8d549b9b0f0bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:27.786384 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.787298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a96bb89b-929b-471d-975e-f0a84da6f0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:27.787578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e60c5-a6a9-44e4-9bf6-252e1a33d01b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.787864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e60c5-a6a9-44e4-9bf6-252e1a33d01b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.788139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e60c5-a6a9-44e4-9bf6-252e1a33d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.791176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1074/2176] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:27 2022] DELETE /identity/v3/projects/cf0ee3bd565e4884bd8d549b9b0f0bf8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:27.795246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527720d-b2f2-4482-a745-cd0172552022 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.795632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527720d-b2f2-4482-a745-cd0172552022 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.796018 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527720d-b2f2-4482-a745-cd0172552022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.808875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5d0e60c5-a6a9-44e4-9bf6-252e1a33d01b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:27.813217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5d0e60c5-a6a9-44e4-9bf6-252e1a33d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.821856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3527720d-b2f2-4482-a745-cd0172552022 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:27.823584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3527720d-b2f2-4482-a745-cd0172552022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.872231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:27.891160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1103/2177] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:27.896548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.899402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.899955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.908700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1075/2178] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:27.917374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.917660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.917931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.943350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:27.943350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.943648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:27.943648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:27.943835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1104/2179] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:27.948851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.951205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1076/2180] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:27.956574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.961287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec65e6-e76a-456e-b76d-2aef22332d17 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.965977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.966643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.966873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c719a-3207-40ae-8937-3af7c72e9472 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:27.966981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.967211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.967867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:27.968171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.969774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.971544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:27.972153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:27.972575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:27.972878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:27.973525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:27.973828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:27.975460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:27.985186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1105/2181] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:27.986349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.987789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1077/2182] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:27.991938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:27.996451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7dd1-7874-4735-8042-47c854bf0561 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.001322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.001878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706e78f-d687-4775-bf26-6b8a3098d5ca admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.002001 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.002207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.002476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.003132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.005186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.006510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.007128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.007405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.007678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.008306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.010227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.041205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1106/2183] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:28.044959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1078/2184] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:27 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:28.045994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.047063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.113855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdc256-aca3-48dd-8279-c11497bc5afa admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.118264 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2c692-9447-407a-94b8-167b2ae35ae7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.119183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.119839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.120084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.120358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.121144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.123375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.123675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.124311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.124589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.124856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.125491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.127478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.133871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:28.148667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1079/2185] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:28 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:28.152172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.155304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1107/2186] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:28 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:28.158998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.162859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68719b6-9237-417e-882b-f05b2fcaa989 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.167622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.168233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.168539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.168961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.169076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e4b45-002e-4077-8d56-5f8cbfbfd537 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.169807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.171455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.174064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.174775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.175197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.175563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.176562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.178282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.183311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1080/2187] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:28.187379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.189738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1108/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:28.193848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.199414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938d873-5b6e-4ab2-94f2-9b25790d1d9a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.205243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.205871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.206206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.206452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.206943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fae640-5202-4bfc-b3b0-5e6ff2341105 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.207135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.208735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.211815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.212422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.212693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.212957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.213618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.215207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.220072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1081/2189] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:28.224371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.229692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1109/2190] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:28.230742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.234712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609954f-b472-421b-a17c-72aaecfefa29 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.239723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.240377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.240655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.240918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] PATH_INFO: `/v3/projects/e4e3c3f1e02b42afb3187b3aa324ee60/users/6e5f5e0ffa194ffa8317dc4cebc0ce6b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.241019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc668d57-83c1-4eaa-a1f3-00106e50f7fc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.246235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.246876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.247210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.247480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] PATH_INFO: `/v3/projects/0113bd01866148a19268a30e6e76d4dc/users/b0905e37ac4643b793e1a8d32938479d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.256068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e3c3f1e02b42afb3187b3aa324ee60, user_id=6e5f5e0ffa194ffa8317dc4cebc0ce6b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.258781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87128-0519-47e6-8a40-111acd8ed76b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.267115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0113bd01866148a19268a30e6e76d4dc, user_id=b0905e37ac4643b793e1a8d32938479d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.269849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d5b8f-4562-4a4b-b3d5-d695fb9b021e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.273758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1082/2191] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:28 2022] PUT /identity/v3/projects/e4e3c3f1e02b42afb3187b3aa324ee60/users/6e5f5e0ffa194ffa8317dc4cebc0ce6b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:28.284283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.284874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.285145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.289446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1110/2192] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:28 2022] PUT /identity/v3/projects/0113bd01866148a19268a30e6e76d4dc/users/b0905e37ac4643b793e1a8d32938479d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:28.293066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.293352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.293723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.313532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:28.314733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:28.316115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] MFA Rules not processed for user `6e5f5e0ffa194ffa8317dc4cebc0ce6b`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:28.317648 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:28.368120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:28.379683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1111/2193] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:28.390158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.390347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1083/2194] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:28.397076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:28.402273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:28.403783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:28.453158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf0d84-572f-406f-b679-9a47575ffdd4 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.456855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.457468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.457746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.458019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.465013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54797bd6-0e74-48c4-93fd-49d69dd429e2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:28.467873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.469187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:28.469433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3e940-5e8f-4784-9a1c-603de55eb3eb service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.469805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:28.470148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:28.470433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:28.479365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:28.480138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1112/2195] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:57:28.480869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:28.490373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1084/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:57:33.711049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:33.711626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:33.711840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:33.748100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] MFA Rules not processed for user `b0905e37ac4643b793e1a8d32938479d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:33.749335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:33.831980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1113/2197] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:33.836873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:33.850251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3744c7-3b19-4442-bbc2-479fb622ec1a service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:33.856067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:33.856698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:33.857085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:33.857360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:33.858128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:33.861102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:33.893735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1085/2198] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:33 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:33.897942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:33.909513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41db4-b66c-4f35-b5de-235829cd4735 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:33.974038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:33.974670 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:33.974948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:33.975257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:33.975910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:33.977885 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.000513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1114/2199] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:33 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:34.004209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.015808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d582e4-c699-4713-a412-d973d4711ea6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.020645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.021259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.021535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.021803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.022445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.024059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.035278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1086/2200] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:57:34.039745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.051519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a68b92-cf15-4f75-a075-de9e012e855f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.056643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.057307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.057583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.057850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] PATH_INFO: `/v3/projects/8eb8a799522548ce8cd0e020c1d98044/users/39939131a5e7429289323b5bf47cc4f7/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.109141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb8a799522548ce8cd0e020c1d98044, user_id=39939131a5e7429289323b5bf47cc4f7, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.111903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.120284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.142271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1115/2201] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:34 2022] PUT /identity/v3/projects/8eb8a799522548ce8cd0e020c1d98044/users/39939131a5e7429289323b5bf47cc4f7/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:34.146811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.158259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eaecf-4db8-45ff-8aca-850e97e0c35f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.162786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:34.225047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.225669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.225945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.226213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.226938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.228615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a96c67-937f-469c-aa64-737a6e0b038c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.239431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1087/2202] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:34.243879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:34.255825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e41ab-3697-4d3b-8d62-cd8cac5aa22a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:34.260840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:34.261513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:34.261784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:34.262162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] PATH_INFO: `/v3/projects/8eb8a799522548ce8cd0e020c1d98044/users/39939131a5e7429289323b5bf47cc4f7/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:34.264137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb8a799522548ce8cd0e020c1d98044, user_id=39939131a5e7429289323b5bf47cc4f7, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:34.266794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:34.278283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1116/2203] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:34 2022] PUT /identity/v3/projects/8eb8a799522548ce8cd0e020c1d98044/users/39939131a5e7429289323b5bf47cc4f7/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:36.914823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c622-6aff-436f-ab21-44bb0b745f81 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:36.915934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c622-6aff-436f-ab21-44bb0b745f81 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:36.916553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c622-6aff-436f-ab21-44bb0b745f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:36.996401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9267c622-6aff-436f-ab21-44bb0b745f81 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:36.997798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9267c622-6aff-436f-ab21-44bb0b745f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:37.115370 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1088/2204] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:57:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:37.157707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:37.160931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:37.290877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68d95-7dfb-4abb-9dea-f42a365bafdd admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:37.297799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:37.299115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:37.299558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:37.299977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:37.312624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:37.314463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:37.326727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1117/2205] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:57:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:57:40.510098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcc348-103f-4e72-b5fd-40b2f938e579 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:40.510729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcc348-103f-4e72-b5fd-40b2f938e579 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:40.510729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcc348-103f-4e72-b5fd-40b2f938e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:40.590453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0fbcc348-103f-4e72-b5fd-40b2f938e579 None None] MFA Rules not processed for user `39939131a5e7429289323b5bf47cc4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:40.591732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0fbcc348-103f-4e72-b5fd-40b2f938e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:40.884217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1089/2206] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:40.898245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:40.901025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:41.039954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7797b-23a8-4afe-a8a6-197bb931bec7 service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:41.044389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:41.045347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:41.045725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:41.046230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:41.215623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:41.218141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:41.237677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1118/2207] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:41.530426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:41.531358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:41.531867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:41.552850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:41.554102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:41.561639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1090/2208] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:57:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:41.579323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:41.594118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91a8ff-a55c-4c9f-ad0c-5e21ac35a1fb service glance] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:41.600874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:41.601584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:41.602078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:41.602599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:41.612907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:41.614803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afead694-6be0-4d02-88e8-f21a66bcbede service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:41.625023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1119/2209] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:57:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:57:46.295580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:46.297716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:46.411075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:46.532500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266d4f2-d4b6-49e3-8b86-e6cd0bb9eadc None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:46.551734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:46.554695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:46.556298 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:46.556931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] PATH_INFO: `/v3/users/66b718c35c01416d87bea1876116e56e` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:46.559788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b718c35c01416d87bea1876116e56e)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:46.562250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:46.695728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b718c35c01416d87bea1876116e56e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:46.726288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b718c35c01416d87bea1876116e56e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:46.762357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b718c35c01416d87bea1876116e56e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:46.766610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b718c35c01416d87bea1876116e56e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:46.825776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b718c35c01416d87bea1876116e56e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:46.857666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1091/2210] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:46 2022] DELETE /identity/v3/users/66b718c35c01416d87bea1876116e56e => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:46.859823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:46.860801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:46.861225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:46.907464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:46.909019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:46.931304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:47.088276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1120/2211] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:47.098694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:47.100255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:47.336590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec66fe2-fa76-4cf0-97e1-f6e64cf5749f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:47.344461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:47.345470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:47.345956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:47.346371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:47.358093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:47.359889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e235-3754-4e6b-ac4e-2d92dfccda9d service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:47.375395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1092/2212] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:48.121240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:48.123516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:48.159967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe130a64-a9f1-4fe6-a119-ce6fea6966df None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:48.261153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:48.262198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:48.262991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:48.263595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] PATH_INFO: `/v3/projects/31a58d46765b42ef8fb5bf489971f58f` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:48.265065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a58d46765b42ef8fb5bf489971f58f)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:48.267572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:48.587519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a58d46765b42ef8fb5bf489971f58f', 'request_id': 'req-ad37a6d7-a545-4adf-b76b-5da3d3faf201'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:48.599781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a58d46765b42ef8fb5bf489971f58f', 'request_id': 'req-ad37a6d7-a545-4adf-b76b-5da3d3faf201'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:48.614282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a58d46765b42ef8fb5bf489971f58f', 'request_id': 'req-ad37a6d7-a545-4adf-b76b-5da3d3faf201'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:48.645243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] The token cache is being invalidate because project 31a58d46765b42ef8fb5bf489971f58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:48.645534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:48.646215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad37a6d7-a545-4adf-b76b-5da3d3faf201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:48.652050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1121/2213] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:48 2022] DELETE /identity/v3/projects/31a58d46765b42ef8fb5bf489971f58f => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:48.654290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:48.654818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:48.654916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:48.689492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:48.691266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:48.775679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:48.813653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:48.814201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:48.814586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:48.818607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1093/2214] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:48.858392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:48.860078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:48.880707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1122/2215] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:48.885398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:48.899815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be42051-0fa7-464a-b1c9-0570cdda021b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:48.916665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:48.917380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:48.917851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:48.919070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:48.919070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:48.919621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:48.921294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:48.934852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1094/2216] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:48.940483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:48.958086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ade54-0315-467f-9057-84e298ab2775 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:48.968854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:48.970078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:48.970589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:48.971099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:48.972030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:48.975739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:49.032618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1123/2217] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:48 2022] POST /identity/v3/projects => generated 361 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:49.037534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:49.064974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fd6d-4d54-4d97-a3b6-017132cb89a3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:49.224936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:49.225700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.225979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.226413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:49.227106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:49.232204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:49.266832 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1095/2218] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:49 2022] POST /identity/v3/users => generated 281 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:49.268413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:49.281367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2504d-7bd0-476d-b3b4-de491cf1a5c1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:49.286160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:49.286860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.287236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.287714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:49.288813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:49.290961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:49.307976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1124/2219] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:49.309338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:49.323949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ecf4-fb2e-43bb-8b76-63edcfddca93 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:49.339860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:49.340751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.340751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.341033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:49.343657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:49.345445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613afb1f-c71a-4fca-9995-ae92be068226 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:49.378602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1096/2220] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:49.380440 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:49.405731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9876-7b82-4fb3-ba59-01491a57560a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:49.439177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:49.440439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.441246 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.441821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] PATH_INFO: `/v3/projects/1be6751ebd084481bcca6c17bedda5a4/users/7a4fa4e64e6e43c79e0af983ea8d943a/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:49.487509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be6751ebd084481bcca6c17bedda5a4, user_id=7a4fa4e64e6e43c79e0af983ea8d943a, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:49.491043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:49.532578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1125/2221] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:49 2022] PUT /identity/v3/projects/1be6751ebd084481bcca6c17bedda5a4/users/7a4fa4e64e6e43c79e0af983ea8d943a/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:49.545459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.549598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.550184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:49.632110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] MFA Rules not processed for user `7a4fa4e64e6e43c79e0af983ea8d943a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:49.635610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:49.791065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1097/2222] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:49.838573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:49.841698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:49.897713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:49.951501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544c0b2-3d1d-416d-9a9f-749979f4587e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:49.964051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:49.964903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:49.965407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:49.966403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:50.022062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:50.026329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4abde-999d-4d49-a4de-b4f82d25dd7b service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:50.064929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1126/2223] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:50.091179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:50.205653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:50.243390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfbd8e-87b3-4b60-84a1-718628c3ce7f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:50.255815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:50.263328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:50.263682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:50.263964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] PATH_INFO: `/v3/users/7a4fa4e64e6e43c79e0af983ea8d943a` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:50.265493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4fa4e64e6e43c79e0af983ea8d943a)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:50.267639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:50.374806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4fa4e64e6e43c79e0af983ea8d943a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:50.389273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4fa4e64e6e43c79e0af983ea8d943a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:50.407033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4fa4e64e6e43c79e0af983ea8d943a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:50.407965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4fa4e64e6e43c79e0af983ea8d943a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:50.428309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4fa4e64e6e43c79e0af983ea8d943a'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:50.436982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1098/2224] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:57:50 2022] DELETE /identity/v3/users/7a4fa4e64e6e43c79e0af983ea8d943a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:50.443305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:50.444377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:50.444799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:50.486010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:50.487504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:50.621754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1127/2225] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:50.628991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:50.630754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:50.785085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b70f-e697-48ee-b021-be9bba0895ab admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:50.789838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:50.790532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:50.790881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:50.791242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:50.821822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:50.823297 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b73a20-fcf6-4135-98c1-3ed5944b5dd5 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:50.834562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1099/2226] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:51.277035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:51.278692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:51.313849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093449b-4272-4c96-a4c6-cfb2097c4218 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:51.329628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:51.330296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:51.330548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:51.330805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] PATH_INFO: `/v3/projects/1be6751ebd084481bcca6c17bedda5a4` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:51.332309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be6751ebd084481bcca6c17bedda5a4)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:51.334237 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:51.437991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be6751ebd084481bcca6c17bedda5a4', 'request_id': 'req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:51.451777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be6751ebd084481bcca6c17bedda5a4', 'request_id': 'req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:51.462200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be6751ebd084481bcca6c17bedda5a4', 'request_id': 'req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:51.493222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] The token cache is being invalidate because project 1be6751ebd084481bcca6c17bedda5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:57:51.494063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:51.494908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a6cc7e4d-4dcc-413f-b501-2eeba2660a5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:57:51.497492 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1128/2227] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:57:51 2022] DELETE /identity/v3/projects/1be6751ebd084481bcca6c17bedda5a4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:57:51.504925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:51.505475 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:51.506002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:51.555685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:51.557109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:51.741786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1100/2228] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:57:51.743444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:51.743729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:51.744010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:51.775249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:51.777557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:51.788429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1129/2229] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:51.792416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:51.811343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0f64-1fd0-4cba-b3a7-080bf555e553 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:51.818629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:51.819417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:51.820063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:51.820218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:51.821177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:57:51.821680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:51.823877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:51.826153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:51.857853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:51.858529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1101/2230] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:57:51.884139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c94c8-12d2-4869-96cc-e901490d3519 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:51.889622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:51.890376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:51.890756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:51.891170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:51.891912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:51.894833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:51.965171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1130/2231] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:57:51 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:57:51.969875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:51.986750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e93cd-8773-4d21-acd9-1d9443b89923 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:52.157198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:52.158746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:52.159815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:52.160596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:52.161575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:52.163806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:52.330915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1102/2232] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:57:51 2022] POST /identity/v3/users => generated 289 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:57:52.340879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:52.372719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c1b3-b7a0-439a-ab8a-575ec5e47b86 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:52.391547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:52.392143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:52.392393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:52.392667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:52.393340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:52.395047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:52.461956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1131/2233] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:52.463728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:52.528079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d8694-03b6-412c-953d-3a4a5a066e47 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:52.533539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:52.537966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:52.538251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:52.538529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:52.539230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:52.540879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b1d1-aad3-4dd2-bc14-eb9bc04be311 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:52.559089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1103/2234] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:57:52.561219 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:52.574323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec7dda-b539-4877-a224-b267e34d6a31 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:52.585245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:52.586287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:52.586764 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:52.587269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] PATH_INFO: `/v3/projects/8664eefedf884105aac162def7c7bf8b/users/f5cb923bd04b4d279d461adcfdad7550/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:52.638400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8664eefedf884105aac162def7c7bf8b, user_id=f5cb923bd04b4d279d461adcfdad7550, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:52.644158 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:52.650514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:57:52.678652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1132/2235] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:57:52 2022] PUT /identity/v3/projects/8664eefedf884105aac162def7c7bf8b/users/f5cb923bd04b4d279d461adcfdad7550/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:57:52.694667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136b3c3-66b8-4af4-bebd-9c3d17d3124a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:52.695373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136b3c3-66b8-4af4-bebd-9c3d17d3124a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:52.695887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136b3c3-66b8-4af4-bebd-9c3d17d3124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:52.749849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a136b3c3-66b8-4af4-bebd-9c3d17d3124a None None] MFA Rules not processed for user `f5cb923bd04b4d279d461adcfdad7550`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:52.753020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a136b3c3-66b8-4af4-bebd-9c3d17d3124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:52.889973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1104/2236] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:57:52.927549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:57:52.929521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:53.049490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f866cb-90d3-4258-b20f-9d41a450b033 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:57:53.054463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:57:53.055351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:53.055788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:53.056239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:53.072349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:57:53.074574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baf30c-6b82-40ab-94c1-d4ea612a9c73 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:57:53.107813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1133/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:57:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:57:55.468214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:57:55.468992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:57:55.468992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:57:55.529410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] MFA Rules not processed for user `b0905e37ac4643b793e1a8d32938479d`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:57:55.531325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:57:55.613572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1105/2238] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:57:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:03.083736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=28 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:04.759243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c698-6d6d-48f4-baf1-072ad52fa6a6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:04.759852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c698-6d6d-48f4-baf1-072ad52fa6a6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:04.759956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c698-6d6d-48f4-baf1-072ad52fa6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:04.781946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6ab6c698-6d6d-48f4-baf1-072ad52fa6a6 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:04.783720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6ab6c698-6d6d-48f4-baf1-072ad52fa6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:04.873499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1134/2239] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 00:58:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:04.881369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:04.883537 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:04.894136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:05.002805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1dc04-eb2a-49da-97c8-2af9374478f3 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:05.011261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:05.012420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:05.012747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:05.013032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:05.024587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:05.027047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:05.048176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1106/2240] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 00:58:05.705697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:05.705984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:05.706259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:05.734191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:05.735929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:05.744200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1135/2241] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:05.758575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:05.770964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723460-fec5-4c85-8de0-fcb6f5ad4daa service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:05.780402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:05.781349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:05.781815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:05.782339 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:05.793202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:05.795361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:05.809259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1107/2242] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:58:07.893833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:07.895488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:07.993327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd7ebc-4509-4f07-a2ad-25804c7059ef None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:07.997773 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:07.998437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:07.998730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:07.999059 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] PATH_INFO: `/v3/users/309da81b8e13455a836e0c8435ed6b06` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:08.000212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309da81b8e13455a836e0c8435ed6b06)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:08.002419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:08.095534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309da81b8e13455a836e0c8435ed6b06'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.104084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309da81b8e13455a836e0c8435ed6b06'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.104872 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309da81b8e13455a836e0c8435ed6b06'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.117943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309da81b8e13455a836e0c8435ed6b06'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.129220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-255fcf13-5d58-46ee-a859-0af60d8ed678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309da81b8e13455a836e0c8435ed6b06'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.139504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1136/2243] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:07 2022] DELETE /identity/v3/users/309da81b8e13455a836e0c8435ed6b06 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:08.345048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:08.360965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4262e16-e80e-485c-83cc-614c3490f031 service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:08.452977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:08.453898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:08.454404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:08.454978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] PATH_INFO: `/v3/projects/5b70056e81304145a833c88ea096e76d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:08.456070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b70056e81304145a833c88ea096e76d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:08.458261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:08.564105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b70056e81304145a833c88ea096e76d', 'request_id': 'req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.571232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b70056e81304145a833c88ea096e76d', 'request_id': 'req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.586839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b70056e81304145a833c88ea096e76d', 'request_id': 'req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.603927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] The token cache is being invalidate because project 5b70056e81304145a833c88ea096e76d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:08.604323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.605154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1af93ef8-ac9c-4bb2-96c9-fb5ab2a07e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:08.606973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1108/2244] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:08 2022] DELETE /identity/v3/projects/5b70056e81304145a833c88ea096e76d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:08.612460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:08.612863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:08.613394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:08.636843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:08.638804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:08.667043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:08.818999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1137/2245] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:08.820803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:08.821012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:08.821336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:08.859605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:08.859863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:08.876488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1109/2246] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:58:08.892647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:08.905281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20cccf-694f-414d-a1d0-843a5ba4a3b1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:08.920591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:08.921508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:08.921791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:08.922042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:08.922898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:58:08.923552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:08.925262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:08.968977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1138/2247] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:58:08.973947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:08.984446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ed263-10f8-4301-8e56-2a9a9a82748e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.003827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.005030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.005030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.005030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.005694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.007703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.014548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:09.065861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1110/2248] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:08 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:09.067238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:09.079100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ebed-f9e3-4524-825a-b28e36d72702 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.153472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.154115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.154379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.154650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.155317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.157596 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.189754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1139/2249] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:09 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:09.194170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:09.205224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e83d0-3f24-4ec0-9aa0-f1f4e4aa2d3a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.210140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.210753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.211068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.211348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.212028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.213641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.223954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1111/2250] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:58:09.228537 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:09.239676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fbfe3-7105-47bb-9b3f-b676314ebaf3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.245194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.246278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.246787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.247343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.248227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.252346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03638ae-94b4-4791-8ae6-f44d7747a3a4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.267387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1140/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:09.271787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:09.282382 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef6237-810a-49d9-8a40-0cefc966f97c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.287050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.287719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.288002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.288282 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] PATH_INFO: `/v3/projects/0dfc6c759ef245499fe3f4253fd6aee7/users/8746ec9b17934dcf9772dcf4bde2258b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.304003 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dfc6c759ef245499fe3f4253fd6aee7, user_id=8746ec9b17934dcf9772dcf4bde2258b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.306673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.322798 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1112/2252] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:09 2022] PUT /identity/v3/projects/0dfc6c759ef245499fe3f4253fd6aee7/users/8746ec9b17934dcf9772dcf4bde2258b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:09.333385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.333716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.333993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.363407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] MFA Rules not processed for user `8746ec9b17934dcf9772dcf4bde2258b`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:09.365162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:09.439643 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1141/2253] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:09.455484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:09.459703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:09.521539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac9fa1-89a7-4924-92cb-9ee07c01d44d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:09.525822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:09.526828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:09.527469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:09.528178 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:09.555081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:09.556471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceb073-d05f-411f-bad9-68a68cf18843 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:09.569395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1113/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:13.330125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:13.346018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:13.463892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d2506-245c-4b62-abd1-cd8d8c25e5ee None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:13.468976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:13.470330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:13.470719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:13.471128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] PATH_INFO: `/v3/users/8746ec9b17934dcf9772dcf4bde2258b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:13.472496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8746ec9b17934dcf9772dcf4bde2258b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:13.474569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:13.575748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8746ec9b17934dcf9772dcf4bde2258b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:13.589078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8746ec9b17934dcf9772dcf4bde2258b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:13.590285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8746ec9b17934dcf9772dcf4bde2258b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:13.610302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8746ec9b17934dcf9772dcf4bde2258b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:13.627502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8746ec9b17934dcf9772dcf4bde2258b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:13.640853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1142/2255] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:13 2022] DELETE /identity/v3/users/8746ec9b17934dcf9772dcf4bde2258b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:13.645729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:13.646726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:13.646726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:13.681252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:13.682480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:13.776219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1114/2256] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:13.799809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:13.801734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:13.897377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f12399-7200-4163-be5e-79b2b91cf6e0 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:13.901598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:13.902232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:13.902510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:13.902789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:13.915912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:13.917588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70a2ab-9155-4178-b5d7-cd7a28a3d47b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:13.930522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1143/2257] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:14.464624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:14.469431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:14.482768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a6e9-38f7-4a31-be2d-5fad8db22d49 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:14.490948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:14.491722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.492329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.496229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] PATH_INFO: `/v3/projects/0dfc6c759ef245499fe3f4253fd6aee7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.499507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dfc6c759ef245499fe3f4253fd6aee7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:14.500879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:14.567469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:14.569204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dfc6c759ef245499fe3f4253fd6aee7', 'request_id': 'req-b7031072-97af-4aec-b4cd-0c723a86f29b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:14.580286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dfc6c759ef245499fe3f4253fd6aee7', 'request_id': 'req-b7031072-97af-4aec-b4cd-0c723a86f29b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:14.589305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dfc6c759ef245499fe3f4253fd6aee7', 'request_id': 'req-b7031072-97af-4aec-b4cd-0c723a86f29b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:14.601946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] The token cache is being invalidate because project 0dfc6c759ef245499fe3f4253fd6aee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:14.602255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:14.602871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b7031072-97af-4aec-b4cd-0c723a86f29b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:14.604481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1115/2258] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:14 2022] DELETE /identity/v3/projects/0dfc6c759ef245499fe3f4253fd6aee7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:14.610280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.610823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.610823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.631349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:14.632504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:14.666307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:14.707278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1144/2259] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:14.710811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.711362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.711484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.737039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:14.738271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:14.750879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1116/2260] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:14.751787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:14.762531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4216d-c137-4b35-8465-238d1df46b62 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:14.770620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:14.771427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.771815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.772154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.772882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:58:14.773245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:14.774970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:14.789466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1145/2261] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:58:14.794159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:14.803714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0bc2d-90d2-4467-9a65-151e8957dff1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:14.808572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:14.809168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.809451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.809722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.810354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:14.812278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:14.848804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1117/2262] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:14 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:14.853110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:14.866362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5151-a17d-4b4f-a46a-eab2bd3a485e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:14.958951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:14.959622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:14.959895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:14.960187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:14.960849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:14.962765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:14.991047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1146/2263] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:14 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:14.994280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.007578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e62a-c8d1-4d55-a257-296cf9f1fe4a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.013048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.013720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.013999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.014266 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.014912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.016866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.027179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1118/2264] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:15.031387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.042326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7e5c-e2f9-4e74-b435-dba7667ddeac admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.047097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.047763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.048050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.048323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.048974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.050585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a7d2-65fd-4190-9d10-8d281edacd47 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.062287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1147/2265] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:15.067626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.078351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e79a0-9661-446c-ba1b-58b2eb7865b2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.083721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.084420 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.084734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.084936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] PATH_INFO: `/v3/projects/2b0736ff08ae4fa6be73ec678c98077f/users/1558d42f4c7f437495af05d4c8c42e6e/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.101569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0736ff08ae4fa6be73ec678c98077f, user_id=1558d42f4c7f437495af05d4c8c42e6e, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.104281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.119944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1119/2266] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:15 2022] PUT /identity/v3/projects/2b0736ff08ae4fa6be73ec678c98077f/users/1558d42f4c7f437495af05d4c8c42e6e/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:15.125164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.125796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.126232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.145744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:15.146876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:15.206860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1148/2267] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:15.216845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.218559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:15.278716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958770ed-ab48-4bcf-a6a6-1616dc4abea7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.282401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.283036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.283328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.283576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.292661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.294094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7724b-0d76-490b-8362-d59e16417474 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.304314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1120/2268] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:58:15.559653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.561374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:15.575097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdac3d-57a0-45b7-932b-03f7a85ed7c9 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.581159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.581894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.582298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.582423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] PATH_INFO: `/v3/users/f5cb923bd04b4d279d461adcfdad7550` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.583686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5cb923bd04b4d279d461adcfdad7550)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.586320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.642148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:15.653190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5cb923bd04b4d279d461adcfdad7550'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:15.662331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5cb923bd04b4d279d461adcfdad7550'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:15.663518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5cb923bd04b4d279d461adcfdad7550'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:15.680455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5cb923bd04b4d279d461adcfdad7550'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:15.692767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5cb923bd04b4d279d461adcfdad7550'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:15.708559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1149/2269] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:15 2022] DELETE /identity/v3/users/f5cb923bd04b4d279d461adcfdad7550 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:15.715688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.716532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.716968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.738929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:15.740468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:15.824983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1121/2270] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:15.839965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:15.842472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:15.937419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6e210-dfed-46d7-9d7f-230cd44c4793 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:15.941799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:15.942549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:15.943262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:15.943574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:15.957339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:15.960062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa1f14-e033-43e0-b12e-b32857096782 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:15.974850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1150/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:16.749916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:16.751769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:16.768949 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859646e7-2c8d-43ac-926c-4cb911529baa None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:16.784269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:16.785231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:16.785668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:16.786114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] PATH_INFO: `/v3/projects/8664eefedf884105aac162def7c7bf8b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:16.790791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8664eefedf884105aac162def7c7bf8b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:16.792773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:16.805627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:16.853822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8664eefedf884105aac162def7c7bf8b', 'request_id': 'req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:16.863548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8664eefedf884105aac162def7c7bf8b', 'request_id': 'req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:16.873112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8664eefedf884105aac162def7c7bf8b', 'request_id': 'req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:16.889664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] The token cache is being invalidate because project 8664eefedf884105aac162def7c7bf8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:16.890034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:16.890950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3a5a4e4c-ca9a-4f90-a894-d0caf10a6e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:16.894335 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1122/2272] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:16 2022] DELETE /identity/v3/projects/8664eefedf884105aac162def7c7bf8b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:16.899917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:16.900498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:16.900632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:16.924204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:16.925533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:17.007959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1151/2273] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:17.009705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.010167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.010469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.036718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:17.038746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:17.063868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1123/2274] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:17.069627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.092657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb9842-946b-40fa-8dac-b06c2c81434a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.097762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.098424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.098746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.098954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.099900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:58:17.100205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.101831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.118413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1152/2275] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:58:17.119825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.134849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824aab8-8c16-4b47-b37b-da2284ee4a6a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.140657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.142045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.143294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.143802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.145364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.149937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.209115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1124/2276] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:17 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:17.213305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.237487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8add-5fc0-4767-9557-333854b304a7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.297781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:17.323417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.324358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.324862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.325561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.326483 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.328727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.361993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1153/2277] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:17 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:17.363288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.374084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb0878-4df1-4c4f-87d3-79c95cb02a8f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.379464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.380084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.380357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.381273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.381529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.385989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.398046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1125/2278] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:17.402275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.414520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51bd0-8c2d-4f33-a068-2a15da8fd590 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.426527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.427199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.427476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.427749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.428424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.430064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893b3b-655d-4046-84d7-1eb2ae10da65 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.442901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1154/2279] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:17.446967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.459539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18586c8c-2517-4e5c-ba12-80c3b516a0c5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.465167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.466385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.466755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.467091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] PATH_INFO: `/v3/projects/ceebf1612c6045b39ce7edddf9c405c2/users/33e857f72d2e4f2299aeb30de8ef42b9/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.494380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceebf1612c6045b39ce7edddf9c405c2, user_id=33e857f72d2e4f2299aeb30de8ef42b9, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.523490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.559445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1126/2280] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:17 2022] PUT /identity/v3/projects/ceebf1612c6045b39ce7edddf9c405c2/users/33e857f72d2e4f2299aeb30de8ef42b9/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:17.561419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.561690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.561933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.649224 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] MFA Rules not processed for user `33e857f72d2e4f2299aeb30de8ef42b9`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:17.650615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:17.808055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1155/2281] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:17.823425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:17.826442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:17.898767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e33d9f-9412-4363-9f86-0b120d14570e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:17.903381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:17.904094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:17.904436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:17.904766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:17.917481 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:17.920278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db984-07bd-46f4-a7a8-310ba5da0665 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:17.933717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1127/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:21.245348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:21.329991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2894-794f-4da3-8594-20a9e110cbb0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:21.334838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:21.335621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:21.336103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:21.336574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] PATH_INFO: `/v3/users/33e857f72d2e4f2299aeb30de8ef42b9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:21.337840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e857f72d2e4f2299aeb30de8ef42b9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:21.339944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:21.409648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e857f72d2e4f2299aeb30de8ef42b9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:21.415719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:21.422626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e857f72d2e4f2299aeb30de8ef42b9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:21.425362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e857f72d2e4f2299aeb30de8ef42b9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:21.443470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e857f72d2e4f2299aeb30de8ef42b9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:21.464002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e857f72d2e4f2299aeb30de8ef42b9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:21.476241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1156/2283] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:21 2022] DELETE /identity/v3/users/33e857f72d2e4f2299aeb30de8ef42b9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:21.481108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104be8d-9e16-4297-a5ee-bf4af1c6a7bf None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:21.481604 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104be8d-9e16-4297-a5ee-bf4af1c6a7bf None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:21.481714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104be8d-9e16-4297-a5ee-bf4af1c6a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:21.504621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2104be8d-9e16-4297-a5ee-bf4af1c6a7bf None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:21.506101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2104be8d-9e16-4297-a5ee-bf4af1c6a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:21.550295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:21.585242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1128/2284] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:21.596197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:21.598835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:21.693032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d382e8-c98b-4f07-8dd3-9ac6ccb4afd0 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:21.711861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:21.715135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:21.715135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:21.715135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:21.740928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:21.742436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:21.780477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1157/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:22.188138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.188439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.188785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.229984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] MFA Rules not processed for user `1558d42f4c7f437495af05d4c8c42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:22.231259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:22.368801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1129/2286] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:22.373348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.374874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:22.389496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7f3a-913e-4292-baf9-e2ca15afdd66 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:22.416303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:22.417288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.417629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.417967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.419414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:22.424025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:22.495034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1158/2287] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:22 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:22.495397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.528251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39609e-3e06-4d6b-baca-e5e255d98dc6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:22.662642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.662642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:22.692632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:22.695353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.696319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.696600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.698284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:22.706568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:22.776855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1130/2288] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:22 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:22.778682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.807951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc41e4-80ae-4426-8b40-3d88472250fc admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:22.811756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e42e5e-3767-47fb-aa9e-ac68c735a9b9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:22.821094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:22.822246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.823074 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:22.823547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.824282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.824657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.825613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:22.825989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.826364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] PATH_INFO: `/v3/projects/ceebf1612c6045b39ce7edddf9c405c2` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.828107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:22.831136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceebf1612c6045b39ce7edddf9c405c2)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:22.834178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:22.856790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1131/2289] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:22.859521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.879075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dd88c-7632-46af-afdb-0010ef9ac630 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:22.886170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:22.888057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:22.889406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:22.889988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] PATH_INFO: `/v3/projects/8dd1826a770746cd93907aa70e4c3aa9/users/62ec214672f54d089fce6dd5f7af19f1/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:22.930187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceebf1612c6045b39ce7edddf9c405c2', 'request_id': 'req-e6ecce86-20ce-4308-8162-c5c737b9cb3a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:22.937884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd1826a770746cd93907aa70e4c3aa9, user_id=62ec214672f54d089fce6dd5f7af19f1, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:22.946503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceebf1612c6045b39ce7edddf9c405c2', 'request_id': 'req-e6ecce86-20ce-4308-8162-c5c737b9cb3a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:22.949519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:22.950126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:22.985543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1132/2290] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:22 2022] PUT /identity/v3/projects/8dd1826a770746cd93907aa70e4c3aa9/users/62ec214672f54d089fce6dd5f7af19f1/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:22.989301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:22.993672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceebf1612c6045b39ce7edddf9c405c2', 'request_id': 'req-e6ecce86-20ce-4308-8162-c5c737b9cb3a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:23.016381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] The token cache is being invalidate because project ceebf1612c6045b39ce7edddf9c405c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:23.017809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:23.019732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e6ecce86-20ce-4308-8162-c5c737b9cb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:23.022893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1159/2291] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:22 2022] DELETE /identity/v3/projects/ceebf1612c6045b39ce7edddf9c405c2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:23.029212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45043a2e-cee2-4580-88ab-01a2dde091aa None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.031422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45043a2e-cee2-4580-88ab-01a2dde091aa None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.032450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45043a2e-cee2-4580-88ab-01a2dde091aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.042272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:23.066835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-45043a2e-cee2-4580-88ab-01a2dde091aa None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:23.068094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-45043a2e-cee2-4580-88ab-01a2dde091aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:23.120819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42b86f-64a0-49da-bfc5-f2c8ae876a92 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.128920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.129519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.129740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.130136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.130800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.133346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.154462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1133/2292] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:23.155655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.158529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:23.179538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180035e-caef-4f41-9743-b9624385c2c7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.186212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.188212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.188789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.189494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] PATH_INFO: `/v3/projects/8dd1826a770746cd93907aa70e4c3aa9/users/62ec214672f54d089fce6dd5f7af19f1/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.207953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd1826a770746cd93907aa70e4c3aa9, user_id=62ec214672f54d089fce6dd5f7af19f1, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.214791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.220363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1160/2293] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:23.222427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.222959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.223250 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.249173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:23.251255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:23.256784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1134/2294] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:23 2022] PUT /identity/v3/projects/8dd1826a770746cd93907aa70e4c3aa9/users/62ec214672f54d089fce6dd5f7af19f1/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:23.399120 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1161/2295] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:23.403613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.425574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d4767-2174-4c65-9a39-3548c16a9c26 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.446506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.447344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.447611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.447894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.448549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:58:23.448920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.451172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.471068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1135/2296] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:58:23.474179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.486785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf1733-76bb-4eb0-ab44-f690e48f3eb5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.493731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.494472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.494854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.495400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.496180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.498534 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.549338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.549912 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1162/2297] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:23 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:23.561818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4511-563e-451b-bb95-621b279ac4b3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.655514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.656141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.656577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.656825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.658181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.660063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.712100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.728060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1136/2298] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:23 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:23.730521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a3220-6c16-4e82-9dd4-09c1e98a099e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.745994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.745994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.746258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.746357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.746958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.750560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.781539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1163/2299] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:58:23.787206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.804341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df101c-c324-4764-a550-5826830af8c3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.811121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.811837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.812186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.812589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.813624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.816512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39a1c6-f690-416f-ad71-f30bfad71c7a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.840266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:23.850670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1137/2300] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:23.859172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3627b9-ccd1-4751-b74a-e6a2fa72e31a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:23.865149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:23.866132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.866411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.866687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] PATH_INFO: `/v3/projects/d4baca9d3a3e45d883482957e9ce8cb0/users/455284c947254207a719459efc031bea/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:23.913287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4baca9d3a3e45d883482957e9ce8cb0, user_id=455284c947254207a719459efc031bea, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:23.917274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:23.960418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320eff-cb3a-4f6b-b465-eedb449d57ae None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:23.960665 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1164/2301] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:23 2022] PUT /identity/v3/projects/d4baca9d3a3e45d883482957e9ce8cb0/users/455284c947254207a719459efc031bea/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:23.960792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320eff-cb3a-4f6b-b465-eedb449d57ae None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:23.960962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320eff-cb3a-4f6b-b465-eedb449d57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:24.011151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8c320eff-cb3a-4f6b-b465-eedb449d57ae None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:24.017169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8c320eff-cb3a-4f6b-b465-eedb449d57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:24.174703 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1138/2302] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:58:24.216033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:24.218833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:24.267809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:24.310178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5213-5646-4c16-bbd5-b9ce5da90d26 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:24.316848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:24.320082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:24.320583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:24.321071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:24.352556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:24.357716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:24.381576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1165/2303] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:24.482380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:24.482819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:24.483221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:24.522672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:24.524119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:24.540232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:24.628811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1139/2304] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:58:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:24.653314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:24.658106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:24.811450 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1ad5d-2768-4a82-b792-7157138f2677 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:24.820980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:24.821636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:24.821919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:24.822191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:24.846625 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:24.849452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f1781-613d-4c22-9541-ad1f3d4df99e service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:24.859871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1166/2305] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:58:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:58:25.306226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:25.308571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:25.338647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a31be-0b01-4bac-9a91-83ea1d8f5343 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:25.345661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:25.346757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:25.347293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:25.347792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] PATH_INFO: `/v3/users/6e5f5e0ffa194ffa8317dc4cebc0ce6b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:25.360328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5f5e0ffa194ffa8317dc4cebc0ce6b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:25.373342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:25.471868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5f5e0ffa194ffa8317dc4cebc0ce6b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:25.485218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5f5e0ffa194ffa8317dc4cebc0ce6b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:25.511237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5f5e0ffa194ffa8317dc4cebc0ce6b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:25.512249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5f5e0ffa194ffa8317dc4cebc0ce6b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:25.541436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5f5e0ffa194ffa8317dc4cebc0ce6b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:25.561723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1140/2306] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:25 2022] DELETE /identity/v3/users/6e5f5e0ffa194ffa8317dc4cebc0ce6b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:25.563834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef116a-9e7d-4b15-a4c1-dd3249a5fc63 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:25.564381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef116a-9e7d-4b15-a4c1-dd3249a5fc63 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:25.565230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef116a-9e7d-4b15-a4c1-dd3249a5fc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:25.616804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-feef116a-9e7d-4b15-a4c1-dd3249a5fc63 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:25.618072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-feef116a-9e7d-4b15-a4c1-dd3249a5fc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:25.897837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1167/2307] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:26.044038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:26.046279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:26.197125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e92e6-663e-45c6-a2c1-39fa69792542 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:26.213081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:26.213778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:26.214062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:26.214464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:26.267968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:26.269445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:26.317773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1141/2308] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:58:27.381045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f0364-892e-43ae-98ee-c1030c11cfd2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:27.381671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f0364-892e-43ae-98ee-c1030c11cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:27.381802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f0364-892e-43ae-98ee-c1030c11cfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:27.420046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-896f0364-892e-43ae-98ee-c1030c11cfd2 None None] MFA Rules not processed for user `1d1c13089bd14d48ba637468182eee14`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:27.424226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-896f0364-892e-43ae-98ee-c1030c11cfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:27.605883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1168/2309] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:28.141668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:28.143287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:28.166974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d0a48-fbf2-48f2-98fc-0443ac92f7ed service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:28.186345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:28.187416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:28.188158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:28.188636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] PATH_INFO: `/v3/projects/e4e3c3f1e02b42afb3187b3aa324ee60` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:28.191573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e3c3f1e02b42afb3187b3aa324ee60)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:28.194379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:28.316986 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:28.357429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e3c3f1e02b42afb3187b3aa324ee60', 'request_id': 'req-6e244702-608e-4e06-a193-164ec2ac881b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:28.369443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e3c3f1e02b42afb3187b3aa324ee60', 'request_id': 'req-6e244702-608e-4e06-a193-164ec2ac881b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:28.396489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e3c3f1e02b42afb3187b3aa324ee60', 'request_id': 'req-6e244702-608e-4e06-a193-164ec2ac881b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:28.441239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] The token cache is being invalidate because project e4e3c3f1e02b42afb3187b3aa324ee60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:28.441239 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:28.442535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-6e244702-608e-4e06-a193-164ec2ac881b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:28.451130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1142/2310] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:28 2022] DELETE /identity/v3/projects/e4e3c3f1e02b42afb3187b3aa324ee60 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:28.453608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:28.454125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:28.454215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:28.468652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:28.516117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:28.517659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:28.767943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1169/2311] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:28.768696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:28.768984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:28.769258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:28.797793 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:28.799662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:28.815984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1143/2312] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:28.821736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:28.839114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d03182-9150-472d-b30b-62d4f0ff4415 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:28.848357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:28.849266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:28.849543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:28.849814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:28.850482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:58:28.850790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:28.852605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:28.899614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1170/2313] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 00:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:58:28.902902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:28.936622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf97c64-290c-4742-a4fb-a8234bd71017 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:28.949471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:28.950544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:28.951930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:28.952445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:28.953962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:28.956588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.012476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1144/2314] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:28 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:29.013770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:29.031696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1757-df3d-42ae-af0f-e19e2a9b7ab3 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:29.131877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:29.132637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.133038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.133468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.134291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:29.136692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.173999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1171/2315] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:29 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:58:29.178811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:29.192923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443daec4-24c1-4e97-98ea-b4ab4b1fc59e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:29.206281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:29.207395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.207953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.209778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.212740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:29.216637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.248641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:29.256133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1145/2316] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:29.266310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65281fec-760c-4ead-89dd-eec02f42fa5a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:29.277577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:29.278228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.278528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.278777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.282015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:29.287977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13b791-97cf-44b0-95f8-e349bb3cd7be admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.338475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1172/2317] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:29.345385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:29.388857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eea338-b82c-4e4d-b083-78c7a7895129 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:29.416259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:29.421605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.421963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.422319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] PATH_INFO: `/v3/projects/b7fdd00d7820471ab9fcfad98240cce2/users/f8840d6738664a59984e9f6305251f07/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.503032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7fdd00d7820471ab9fcfad98240cce2, user_id=f8840d6738664a59984e9f6305251f07, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:29.503032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.531496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1146/2318] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:29 2022] PUT /identity/v3/projects/b7fdd00d7820471ab9fcfad98240cce2/users/f8840d6738664a59984e9f6305251f07/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:58:29.534353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6189937-9098-40db-b7ef-7959f73204d9 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.535291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6189937-9098-40db-b7ef-7959f73204d9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.535743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6189937-9098-40db-b7ef-7959f73204d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.608075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a6189937-9098-40db-b7ef-7959f73204d9 None None] MFA Rules not processed for user `f8840d6738664a59984e9f6305251f07`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:29.608075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a6189937-9098-40db-b7ef-7959f73204d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:29.714857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:29.730807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1173/2319] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:29.738831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:29.742012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:29.860802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bfbdc-946d-4d5c-8cc8-853084e75327 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:29.864788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:29.865590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:29.868794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:29.868794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:29.880458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:29.882120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:29.894899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1147/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 00:58:31.983282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c3c45-d467-4aa4-8ab9-95f7ab541b27 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:31.983745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c3c45-d467-4aa4-8ab9-95f7ab541b27 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:31.983905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c3c45-d467-4aa4-8ab9-95f7ab541b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:32.076048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-324c3c45-d467-4aa4-8ab9-95f7ab541b27 None None] MFA Rules not processed for user `62ec214672f54d089fce6dd5f7af19f1`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:32.077540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-324c3c45-d467-4aa4-8ab9-95f7ab541b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:32.347311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1174/2321] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:32.831332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:32.834048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:32.908098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:32.941758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667acb0-02ec-4bb4-82af-99eafd4ef45d service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:32.949707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:32.950689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:32.951122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:32.951499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:32.954065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:33.061071 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:33.064150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d98a81-b874-4d60-a796-ffb422f9221a service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:33.090153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1148/2322] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:35.692350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:35.693793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:35.694286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:35.753345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] MFA Rules not processed for user `1558d42f4c7f437495af05d4c8c42e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:35.754775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:35.773183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1175/2323] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:41.175113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:41.175113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:41.175113 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:41.228661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] MFA Rules not processed for user `455284c947254207a719459efc031bea`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:41.232846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:41.349414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1149/2324] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 00:58:41.353850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:41.355552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:41.515475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04c000-1369-4f2a-99c8-b6e7d4e0ddac None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:41.520777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:41.522100 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:41.522595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:41.523236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:41.524187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:41.526856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:41.627314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1176/2325] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 00:58:41 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:41.631734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:41.648731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baeb07-0c53-49dc-8a03-dfbc19036a0b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:41.744634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:41.745346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:41.745552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:41.746316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:41.746972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:41.752870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:41.798524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:41.799733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1150/2326] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 00:58:41 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:58:41.811530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322356b9-dbea-4460-8f06-fb9ac04fb415 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:41.828098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:41.829017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:41.829473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:41.829999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:41.830856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:41.832909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:41.851938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1177/2327] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:41.853879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:41.871706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b0c74-8e0e-4a84-bb7f-053c49492850 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:41.884017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:41.884729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:41.885732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:41.886064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] PATH_INFO: `/v3/projects/9462ffe85319404bb2bb8be3b9855efa/users/74d3fcced3ff48f09e6f202127c503f4/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:41.892882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:41.934858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9462ffe85319404bb2bb8be3b9855efa, user_id=74d3fcced3ff48f09e6f202127c503f4, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:41.935895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:41.956640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1151/2328] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:41 2022] PUT /identity/v3/projects/9462ffe85319404bb2bb8be3b9855efa/users/74d3fcced3ff48f09e6f202127c503f4/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:41.960658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:41.976169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b62477-6496-4222-9e8e-d61527b17ba1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:42.041216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:42.061415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:42.062049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:42.062382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:42.062653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:42.063557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:42.065801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:42.080018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1178/2329] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 00:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:58:42.081342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:42.092400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f8764-7230-4517-aec4-ac47a04f29fe admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:42.099538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:42.100220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:42.100494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:42.100769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] PATH_INFO: `/v3/projects/9462ffe85319404bb2bb8be3b9855efa/users/74d3fcced3ff48f09e6f202127c503f4/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:42.102760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9462ffe85319404bb2bb8be3b9855efa, user_id=74d3fcced3ff48f09e6f202127c503f4, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:42.106586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:42.127242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1152/2330] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 00:58:42 2022] PUT /identity/v3/projects/9462ffe85319404bb2bb8be3b9855efa/users/74d3fcced3ff48f09e6f202127c503f4/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:49.524880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:49.526882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:49.596383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0614-2f27-4e53-926b-a062fee8322c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:49.666515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:49.667194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:49.667469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:49.667995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:49.669914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:49.794461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:49.796049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976432e-3f9f-47a8-a834-6964615f5aaa service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:49.811485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1179/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:49.964678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:49.967672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:50.075658 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c017f-0c56-43a7-9d48-a3a4fc0f62aa admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:50.096350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:50.097471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:50.097752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:50.098032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:50.099988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:50.115418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:50.116901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:50.129533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1153/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:50.637619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:50.638139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:50.638345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:50.675913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] MFA Rules not processed for user `74d3fcced3ff48f09e6f202127c503f4`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:50.677494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:50.747527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:50.791369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1180/2333] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:50.811220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:50.814373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:50.980112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd42-416c-4125-8cd7-235bed857b5b service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:50.986778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:50.987458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:50.987739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:50.988012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:51.049572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:51.051130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d6b27-f1fa-4045-9288-69e0f49aaee8 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:51.056249 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:51.063106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1154/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:51.620762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:51.721511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbace9d8-0f02-421c-9ffa-8d0b007a7612 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:51.728485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:51.729455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:51.730289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:51.730867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] PATH_INFO: `/v3/users/f8840d6738664a59984e9f6305251f07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:51.732278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8840d6738664a59984e9f6305251f07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:51.734672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:51.817536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8840d6738664a59984e9f6305251f07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:51.830850 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8840d6738664a59984e9f6305251f07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:51.832433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8840d6738664a59984e9f6305251f07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:51.857358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8840d6738664a59984e9f6305251f07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:51.873474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8840d6738664a59984e9f6305251f07'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:51.881244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1181/2335] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 00:58:51 2022] DELETE /identity/v3/users/f8840d6738664a59984e9f6305251f07 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:51.885727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:51.886013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:51.886290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:51.913904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:51.915321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:52.004305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1155/2336] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:52.014190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:52.015856 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:52.090274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cd26c-4e9a-42d1-842f-698583c414b2 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:52.094263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:52.095194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:52.095468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:52.095739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:52.107661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:52.110424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4598f-d1c2-420d-b96d-8c36bf6e8b53 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:52.121570 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1182/2337] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:52.704439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:52.705941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:52.733728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ec23-4853-4c11-906e-98ac9eaad2a7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:52.741865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:52.742898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:52.743429 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:52.743920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] PATH_INFO: `/v3/projects/b7fdd00d7820471ab9fcfad98240cce2` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:52.745601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7fdd00d7820471ab9fcfad98240cce2)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:52.748686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:52.838483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7fdd00d7820471ab9fcfad98240cce2', 'request_id': 'req-22c59589-cb44-4070-adf3-ebf72de37e89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:52.847357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7fdd00d7820471ab9fcfad98240cce2', 'request_id': 'req-22c59589-cb44-4070-adf3-ebf72de37e89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:52.867507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7fdd00d7820471ab9fcfad98240cce2', 'request_id': 'req-22c59589-cb44-4070-adf3-ebf72de37e89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:52.896349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] The token cache is being invalidate because project b7fdd00d7820471ab9fcfad98240cce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 00:58:52.896600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:52.897396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-22c59589-cb44-4070-adf3-ebf72de37e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 00:58:52.900915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1156/2338] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 00:58:52 2022] DELETE /identity/v3/projects/b7fdd00d7820471ab9fcfad98240cce2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:58:55.126836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.127317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.127411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.129013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1183/2339] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:58:55.135461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0cfb1-1f9d-4b6b-ba0f-04409662121d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.136006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0cfb1-1f9d-4b6b-ba0f-04409662121d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.136006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0cfb1-1f9d-4b6b-ba0f-04409662121d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.181408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-48c0cfb1-1f9d-4b6b-ba0f-04409662121d None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:55.184198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-48c0cfb1-1f9d-4b6b-ba0f-04409662121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:55.273491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1157/2340] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:58:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:55.286893 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:55.289438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:55.299745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:55.381383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1af03-a0ba-4d5c-826d-d1d3b8908a22 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:55.385421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:55.386144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.386382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.386678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.399265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:55.400736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab125205-6e52-487c-b78b-da0864decdc7 service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:55.414573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1184/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:55.564275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.564558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.565008 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.566494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1158/2342] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:58:55.579399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df98-9489-42cb-94ac-9d4993faa847 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.580003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df98-9489-42cb-94ac-9d4993faa847 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.580498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df98-9489-42cb-94ac-9d4993faa847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.619336 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-efa5df98-9489-42cb-94ac-9d4993faa847 None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:55.620551 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-efa5df98-9489-42cb-94ac-9d4993faa847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:55.693022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1185/2343] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:58:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:55.709712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:55.711726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:55.733243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e0f1d-d914-4a43-bdd7-415d4594e0ad None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:55.744491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:55.745119 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.745396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.745664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.756360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:55.758174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265811-c5e5-48cd-b02a-81907d7cd9a7 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:55.766204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:58:55.803834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1159/2344] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:55.934897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432685-9457-4eb4-8b4a-16bb8a95d8da None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.935279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432685-9457-4eb4-8b4a-16bb8a95d8da None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.935488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432685-9457-4eb4-8b4a-16bb8a95d8da None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.937029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1186/2345] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:58:55.943296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601ae6-4581-4a44-a62a-d1f00406cf31 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:55.943490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601ae6-4581-4a44-a62a-d1f00406cf31 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:55.943782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601ae6-4581-4a44-a62a-d1f00406cf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:55.976990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1d601ae6-4581-4a44-a62a-d1f00406cf31 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:55.978620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1d601ae6-4581-4a44-a62a-d1f00406cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:55.991906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1160/2346] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 00:58:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:56.004950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c432685-9457-4eb4-8b4a-16bb8a95d8da None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:56.022448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c432685-9457-4eb4-8b4a-16bb8a95d8da None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:56.029326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:56.029968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:56.030249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:56.030668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:56.049448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:56.051031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:56.085894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1187/2347] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:58:59.292524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885237a-6ba0-49b4-b812-8e43b1e4af2a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:59.293168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885237a-6ba0-49b4-b812-8e43b1e4af2a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:59.293399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885237a-6ba0-49b4-b812-8e43b1e4af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:59.323046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5885237a-6ba0-49b4-b812-8e43b1e4af2a None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:58:59.324449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5885237a-6ba0-49b4-b812-8e43b1e4af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:59.433510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1161/2348] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:58:59.461507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:58:59.464734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:58:59.565584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605079c-a18c-4983-9cc1-d5253d86ab65 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:58:59.570858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:58:59.571530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:58:59.571804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:58:59.572069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:58:59.599095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:58:59.600704 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b50aa-ed7f-464f-bda6-730f41920ba0 service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:58:59.625593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1188/2349] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:58:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:59:09.416851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=28 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:09.607223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:13.137356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:13.137940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:13.137940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:13.158676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] MFA Rules not processed for user `455284c947254207a719459efc031bea`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:13.159956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:13.237216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1162/2350] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 00:59:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:23.230226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=25 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:23.432786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:23.433279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:23.433418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:23.454838 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:59:23.456520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:23.463996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1189/2351] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 00:59:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 00:59:23.480732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:23.492889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459c8be-c2e0-4308-b022-193a1db883ae None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:23.498036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:23.498685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:23.498962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:23.499260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:23.508012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:23.509635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:23.518015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1163/2352] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 00:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:59:28.990848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:28.992569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:29.005050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a8cb-9294-4b50-926a-d0b9452721f9 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:29.009559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:29.010110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:29.010389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:29.010673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:29.012522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:29.088262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:29.089817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245cf51-b05c-4f5c-ac27-0794ad21df8b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:29.103878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1190/2353] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:32.607751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:59:32.618343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74e9cb-b32b-4856-a50b-011bc1c65b5b service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:59:32.623039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:59:32.623617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:59:32.623931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:59:32.624194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:59:32.626224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:59:32.693253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:59:32.694741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6dd49-9a25-4be4-a308-2452c1b8f969 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:59:32.704433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1164/2354] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 00:59:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 00:59:39.097984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=33 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:59:42.700156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=36 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:19.912136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:19.912729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:19.912729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:19.933426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] MFA Rules not processed for user `1d1c13089bd14d48ba637468182eee14`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:19.934624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:20.000000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1191/2355] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:00:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:29.230181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:29.230747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:29.230932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:29.251243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] MFA Rules not processed for user `f2a8b56e48b445709054849f4893767a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:29.252519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:29.310594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1165/2356] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:00:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:29.992522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=25 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:35.966403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:35.977086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b08c8-67fe-4980-82e4-5969ed0fcdc6 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:35.981546 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:35.982153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:35.982428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:35.982831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:35.984835 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:35.999281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:36.000849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62156973-4094-4afe-a6c9-4973b434786e service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:36.018371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1192/2357] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:00:36.258692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:36.269020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685fbf-9dd6-4135-991c-1f387b3d76c9 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:36.273112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:36.273717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:36.273999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:36.274270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:36.276202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:36.290589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:36.292049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e065fb-3934-44f6-b325-f5b693a22c19 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:36.303979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1166/2358] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:00:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:00:44.604147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:44.604733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:44.604733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:44.624429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] MFA Rules not processed for user `22f11e29c4ad4712a54877b0895d28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:44.625641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:44.684277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1193/2359] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:00:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:46.298453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=31 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:00:51.146243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:51.146845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:51.146845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:51.167411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:00:51.168623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:51.177065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1167/2360] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 01:00:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:00:51.230451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:51.243652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ea583-6549-46cf-9bb8-c7f1ddc2ab1d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:51.249136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:51.249737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:51.250013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:51.250262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:51.258304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:51.260078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:51.268253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1194/2361] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:00:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:00:59.489540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.491044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:00:59.584913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0cc3e-1e66-4bcc-a830-9f6f95bde5fd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.593840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:00:59.600010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:00:59.600301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:00:59.600577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] PATH_INFO: `/v3/users/b0905e37ac4643b793e1a8d32938479d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:00:59.602005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0905e37ac4643b793e1a8d32938479d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:00:59.604002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:00:59.669415 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0905e37ac4643b793e1a8d32938479d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:00:59.678910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0905e37ac4643b793e1a8d32938479d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:00:59.691223 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0905e37ac4643b793e1a8d32938479d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:00:59.692111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0905e37ac4643b793e1a8d32938479d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:00:59.708322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0905e37ac4643b793e1a8d32938479d'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:00:59.718029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1168/2362] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:00:59 2022] DELETE /identity/v3/users/b0905e37ac4643b793e1a8d32938479d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:00:59.929549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:00:59.960383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485c9a2-6d90-4c11-86e2-e80bab23448d service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:00:59.997240 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=50 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.031288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.031935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.032351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.032619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] PATH_INFO: `/v3/projects/0113bd01866148a19268a30e6e76d4dc` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.033744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0113bd01866148a19268a30e6e76d4dc)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.035698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.093666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0113bd01866148a19268a30e6e76d4dc', 'request_id': 'req-24da031f-90bb-4874-8670-29ae15aac617'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.106212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0113bd01866148a19268a30e6e76d4dc', 'request_id': 'req-24da031f-90bb-4874-8670-29ae15aac617'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.113561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0113bd01866148a19268a30e6e76d4dc', 'request_id': 'req-24da031f-90bb-4874-8670-29ae15aac617'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.129324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] The token cache is being invalidate because project 0113bd01866148a19268a30e6e76d4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:00.129601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.130296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.131846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1195/2363] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:00:59 2022] DELETE /identity/v3/projects/0113bd01866148a19268a30e6e76d4dc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:00.136076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.137669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:00.154017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcbe5-3ece-49dc-a8b9-224296d45813 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.213421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.214058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.214332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.214602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] PATH_INFO: `/v3/users/39939131a5e7429289323b5bf47cc4f7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.215872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39939131a5e7429289323b5bf47cc4f7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.217929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.231913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.271294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39939131a5e7429289323b5bf47cc4f7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.278137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39939131a5e7429289323b5bf47cc4f7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.287280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39939131a5e7429289323b5bf47cc4f7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.288312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39939131a5e7429289323b5bf47cc4f7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.302797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39939131a5e7429289323b5bf47cc4f7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.309922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1169/2364] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:00 2022] DELETE /identity/v3/users/39939131a5e7429289323b5bf47cc4f7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:00.499677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:00.501333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:00.515473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da031f-90bb-4874-8670-29ae15aac617 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:00.573965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:00.574613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:00.574892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:00.575199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] PATH_INFO: `/v3/projects/8eb8a799522548ce8cd0e020c1d98044` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:00.576456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb8a799522548ce8cd0e020c1d98044)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:00.578616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:00.628783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb8a799522548ce8cd0e020c1d98044', 'request_id': 'req-9ead1a71-e001-4a1b-bf4f-140e630650bb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.634750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb8a799522548ce8cd0e020c1d98044', 'request_id': 'req-9ead1a71-e001-4a1b-bf4f-140e630650bb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.639448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:00.640661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb8a799522548ce8cd0e020c1d98044', 'request_id': 'req-9ead1a71-e001-4a1b-bf4f-140e630650bb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.654931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] The token cache is being invalidate because project 8eb8a799522548ce8cd0e020c1d98044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:00.655260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.655905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:00.657435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1196/2365] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:00 2022] DELETE /identity/v3/projects/8eb8a799522548ce8cd0e020c1d98044 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:04.581061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:04.582728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:04.647028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bf7b-70b5-4fbf-8020-b4928526f7e6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:04.699912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:04.700438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:04.700712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:04.700983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:04.703172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:04.770130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:04.771636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:04.784811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1170/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:10.652280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=3,keystone:INSERT=1,keystone:DELETE=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:14.778981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:18.369428 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:18.371145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.469166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead1a71-e001-4a1b-bf4f-140e630650bb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:18.475258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:18.475888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:18.476218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:18.476581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:18.478638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:18.492568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:18.494225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:18.506707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1197/2367] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:19.268082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:19.282633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c88b-f2ba-4cf5-931e-ee1657fafb10 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:19.290074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:19.290666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:19.290938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:19.291234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:19.293301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:19.356766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:19.358385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:19.368707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1171/2368] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:25.099267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:25.101296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:25.170245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb470ef5-23db-4abc-be0c-0206e6465fe7 service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:25.174837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:25.175489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:25.175775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:25.176049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:25.178104 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:25.244621 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:25.246070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:25.256031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1198/2369] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:28.719869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:28.722086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:28.736879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717664a-8a6d-4789-8bff-ac676a64cc7c service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:28.741966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:28.742602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:28.742878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:28.743190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] PATH_INFO: `/v3/users/1d1c13089bd14d48ba637468182eee14` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:28.744383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1c13089bd14d48ba637468182eee14)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:28.746436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:28.806464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1c13089bd14d48ba637468182eee14'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:28.813961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1c13089bd14d48ba637468182eee14'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:28.823186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1c13089bd14d48ba637468182eee14'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:28.823824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1c13089bd14d48ba637468182eee14'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:28.836976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1b2d5eb1-11b0-4e21-b8dd-5f2bd80cb159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1c13089bd14d48ba637468182eee14'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:28.843271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1172/2370] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:28 2022] DELETE /identity/v3/users/1d1c13089bd14d48ba637468182eee14 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:29.042049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.054691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c143f6-2227-446e-acec-a31c2b08d0cc service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.091634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:29.119506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.120155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.120434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.120706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] PATH_INFO: `/v3/projects/9a1cf0a235aa4669a9276c293a78f2eb` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.121831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1cf0a235aa4669a9276c293a78f2eb)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.123824 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.176910 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1cf0a235aa4669a9276c293a78f2eb', 'request_id': 'req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:29.183711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1cf0a235aa4669a9276c293a78f2eb', 'request_id': 'req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:29.190540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1cf0a235aa4669a9276c293a78f2eb', 'request_id': 'req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:29.203117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] The token cache is being invalidate because project 9a1cf0a235aa4669a9276c293a78f2eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:29.203429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:29.204033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-34bd1e22-8e1b-4312-b8ea-c0ba2845013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:29.205571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1199/2371] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:29 2022] DELETE /identity/v3/projects/9a1cf0a235aa4669a9276c293a78f2eb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:29.210727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.211024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.211296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.233276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:29.234565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:29.304082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1173/2372] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:29.312616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.312897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.313165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.334362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:29.335946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:29.344669 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1200/2373] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:29.350140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.360555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5769ac-f896-4648-89a3-f2cea6d76064 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.366326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.366926 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.367235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.367529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.368170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:29.368471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.370081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.376900 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:29.383586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1174/2374] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:29.388404 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.398555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85bb2d-d9ab-4204-844d-12dd827bd77d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.404677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.405287 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.406048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.406317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.406956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.408886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.442716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1201/2375] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:29.447183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.457517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6fb1-a5bf-48cd-82cd-4552ae90b938 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.518440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.519096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.519364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.519691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.520287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.522242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.545547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1175/2376] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:29.549899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.560906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4ed6f-ff19-4b33-b720-79ac8b1f706b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.565732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.566351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.566631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.566906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.567598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.569457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.580055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1202/2377] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:29.584192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.594804 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327926f-72e1-460b-8a8f-9cfd16ea583e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.599532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.600090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.600503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.600982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.601585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.603210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4680a-75b6-4a70-a0a3-7cbe26628014 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.614759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1176/2378] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:29.619140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.630248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288b3a-b00b-4166-897f-b2c54edb9de1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.634972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.635679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.635955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.636225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] PATH_INFO: `/v3/projects/cd95df36e57745689fe82b4fa211d3ad/users/d4c5c92ff3ea446aa2bffe1255dccfde/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.651118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd95df36e57745689fe82b4fa211d3ad, user_id=d4c5c92ff3ea446aa2bffe1255dccfde, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.653781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.668826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1203/2379] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:29 2022] PUT /identity/v3/projects/cd95df36e57745689fe82b4fa211d3ad/users/d4c5c92ff3ea446aa2bffe1255dccfde/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:29.679865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.680156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.680503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.711775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] MFA Rules not processed for user `d4c5c92ff3ea446aa2bffe1255dccfde`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:29.713140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:29.779916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1177/2380] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:29.784646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.848863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdce20-504b-49d7-bef8-d4c0fd9a35aa admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.853527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.854145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.854420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.854690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.855362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.857301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.862684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:29.887702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1204/2381] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:29.892047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:29.903692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9d53d-f55d-4855-8cbb-04458a5de4b6 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:29.969770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:29.970400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:29.970674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:29.970995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:29.972254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:29.974288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:29.986353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:30.000417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1178/2382] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:29 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:30.004715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.015563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074ca4e-9d8f-4df7-aeed-6aad51e87862 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.020175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.020772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.021044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.021310 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.021981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.023915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.034916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1205/2383] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:30.039363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.050480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5894-ac8e-4dbf-8955-9213c975799a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.056503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.057162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.057447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.057720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] PATH_INFO: `/v3/projects/3f1173827ab94affa87cdc703a309e75/users/9d990aeca5464447b24d8cddffe15736/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.073179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1173827ab94affa87cdc703a309e75, user_id=9d990aeca5464447b24d8cddffe15736, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.075903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.091556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1179/2384] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:30 2022] PUT /identity/v3/projects/3f1173827ab94affa87cdc703a309e75/users/9d990aeca5464447b24d8cddffe15736/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:30.096386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.107193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0bd-a9a3-418e-8ea6-c639974cef2f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.164819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.165435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.165710 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.165982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.166622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.168271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfc58b-cbc3-46eb-a037-0e1f7c4c2526 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.178461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1206/2385] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:30.183175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.195478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33739453-8721-4421-b515-86cbe3ed05cc admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.202656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.203309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.203590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.203868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] PATH_INFO: `/v3/projects/3f1173827ab94affa87cdc703a309e75/users/9d990aeca5464447b24d8cddffe15736/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.208402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1173827ab94affa87cdc703a309e75, user_id=9d990aeca5464447b24d8cddffe15736, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.211134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.222764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1180/2386] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:30 2022] PUT /identity/v3/projects/3f1173827ab94affa87cdc703a309e75/users/9d990aeca5464447b24d8cddffe15736/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:30.234606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.235175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.235521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.273610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] MFA Rules not processed for user `9d990aeca5464447b24d8cddffe15736`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:30.274859 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:30.361191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1207/2387] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:30.372781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.374649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:30.445668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd4ffd-444c-4ea0-9d20-861d9246f027 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.449709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.450342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.450618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.450885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.460366 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.461843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d574b0-25f2-4839-b407-7e0382ceca75 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.471996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1181/2388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:30 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:30.630936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.697485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:30.703752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455ea9d-f439-42cd-be54-0ced7c4b97e7 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.710623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.711376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.711652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.711924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] PATH_INFO: `/v3/users/d4c5c92ff3ea446aa2bffe1255dccfde` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.713073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c5c92ff3ea446aa2bffe1255dccfde)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.715014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.777356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c5c92ff3ea446aa2bffe1255dccfde'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:30.784318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c5c92ff3ea446aa2bffe1255dccfde'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:30.785025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c5c92ff3ea446aa2bffe1255dccfde'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:30.796752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c5c92ff3ea446aa2bffe1255dccfde'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:30.805374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c5c92ff3ea446aa2bffe1255dccfde'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:30.811887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1208/2389] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:30 2022] DELETE /identity/v3/users/d4c5c92ff3ea446aa2bffe1255dccfde => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:30.816521 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.817045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.817152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.837081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:30.838300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:30.899102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1182/2390] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:30.908845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:30.910554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:30.972578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0357dc-3293-4c96-8a37-26340600ed2e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:30.976463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:30.977077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:30.977353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:30.977622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:30.987653 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:30.989108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:30.998951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1209/2391] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:31.282670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:31.284216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:31.298833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041abf5-b2db-4cef-b475-9b4819aa4c5b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:31.304639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:31.305643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:31.305643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:31.305884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] PATH_INFO: `/v3/projects/cd95df36e57745689fe82b4fa211d3ad` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:31.307372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd95df36e57745689fe82b4fa211d3ad)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:31.309794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:31.356323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd95df36e57745689fe82b4fa211d3ad', 'request_id': 'req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.360919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:31.362431 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd95df36e57745689fe82b4fa211d3ad', 'request_id': 'req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.376559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd95df36e57745689fe82b4fa211d3ad', 'request_id': 'req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.388313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] The token cache is being invalidate because project cd95df36e57745689fe82b4fa211d3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:31.388595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.389218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.390683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1183/2392] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:31 2022] DELETE /identity/v3/projects/cd95df36e57745689fe82b4fa211d3ad => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:31.396308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:31.406730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af1fdd-dd07-4e7f-9190-5db6f4839297 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:31.471457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:31.472088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:31.472357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:31.472624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] PATH_INFO: `/v3/users/9d990aeca5464447b24d8cddffe15736` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:31.473796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d990aeca5464447b24d8cddffe15736)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:31.475785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:31.499611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:31.535808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d990aeca5464447b24d8cddffe15736'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.543330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d990aeca5464447b24d8cddffe15736'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.544028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d990aeca5464447b24d8cddffe15736'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.557587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d990aeca5464447b24d8cddffe15736'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.567306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a1c848fa-6c6d-485b-94e7-4b0b98af0bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d990aeca5464447b24d8cddffe15736'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:31.573580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1210/2393] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:01:31 2022] DELETE /identity/v3/users/9d990aeca5464447b24d8cddffe15736 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:31.852135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:31.853713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:31.867678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bda27-45fe-4a5d-a0b7-133a364d5ace admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.058690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.059696 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.059971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.060236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] PATH_INFO: `/v3/projects/3f1173827ab94affa87cdc703a309e75` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.061358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1173827ab94affa87cdc703a309e75)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.063306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.111751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1173827ab94affa87cdc703a309e75', 'request_id': 'req-2044882d-cf79-42aa-9a06-65c683c08016'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:32.118570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1173827ab94affa87cdc703a309e75', 'request_id': 'req-2044882d-cf79-42aa-9a06-65c683c08016'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:32.128101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1173827ab94affa87cdc703a309e75', 'request_id': 'req-2044882d-cf79-42aa-9a06-65c683c08016'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:32.141531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] The token cache is being invalidate because project 3f1173827ab94affa87cdc703a309e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:01:32.141912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:32.142607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2044882d-cf79-42aa-9a06-65c683c08016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:01:32.144167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1184/2394] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:01:31 2022] DELETE /identity/v3/projects/3f1173827ab94affa87cdc703a309e75 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:32.149460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.149776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.150046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.170907 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:32.172155 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:32.233328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1211/2395] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:32.241315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.241600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.241872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.262994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:32.264240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:32.273090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1185/2396] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:32.278937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.291847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e417-05bb-402e-8f5e-17bbb1d358df None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.300318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.301249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.301722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.302178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.303309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:32.303814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.306108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.319380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1212/2397] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:32.324602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.334501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dc02d-c687-44f5-9d2d-270d23a3d6d7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.340996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.341591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.341868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.342136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.342771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.344748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.382646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1186/2398] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:32.387108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.397588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ea0d-265f-4a82-987a-a4e452c103c1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.458128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.458751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.459086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.459361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.459991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.462317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.487999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1213/2399] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:32.493862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.510578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cc290-367d-4bf7-af34-796825a198df admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.516485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.517099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.517360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.518360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.519177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.520794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.531882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1187/2400] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:32.537079 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.552532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7963-b73b-4d54-8c86-d812d289d2ac admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.561204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.562103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.562488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.562880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.563841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.566331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6f3c-cb41-4132-a408-7f682640971d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.579248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1214/2401] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:32.583891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.594655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702daa6-9309-4219-b7b3-ef5cfb69796b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.600732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.601385 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.601675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.601945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] PATH_INFO: `/v3/projects/571676f2f37b40288316568455af5f2b/users/6329687f545c42f7a60dbef46e1a74d3/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.617361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=571676f2f37b40288316568455af5f2b, user_id=6329687f545c42f7a60dbef46e1a74d3, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.620066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.635954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1188/2402] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:32 2022] PUT /identity/v3/projects/571676f2f37b40288316568455af5f2b/users/6329687f545c42f7a60dbef46e1a74d3/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:32.647601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.647876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.648180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.656654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:32.677775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] MFA Rules not processed for user `6329687f545c42f7a60dbef46e1a74d3`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:32.679045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:32.748414 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1215/2403] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:32.753637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.767165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:32.823579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181850ef-a66f-4ff3-8a64-bb3600ebe3a8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.828650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.829294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.829568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.829833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.830468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.832413 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.862765 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1189/2404] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:32.867017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.877948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089874d5-d233-49e5-974a-e52ba29a7603 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.942888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.943526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.943801 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.944068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.944683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.946831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:32.971796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1216/2405] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:01:32 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:32.975870 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:32.986876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d79ddd-66bb-4431-a82f-59a6ef547942 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:32.992800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:32.993688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:32.994087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:32.994363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:32.995031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:32.996680 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.008034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1190/2406] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:33.012571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.022982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41979cb8-8646-4be1-979d-9d1ca0fcbac0 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.028260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.028920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.029197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.029466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] PATH_INFO: `/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4/users/024822935b5f4d8f899f87567e32b070/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.044607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae6dedc52f54fbd8d05195bacaa88a4, user_id=024822935b5f4d8f899f87567e32b070, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.047300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.062922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1217/2407] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:33 2022] PUT /identity/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4/users/024822935b5f4d8f899f87567e32b070/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:33.067094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.077399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad851a-5422-4d20-b6cd-37a81a30ccd5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.133577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.134330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.134607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.134878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.135552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.137153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a5323-13f1-45ac-a781-2dd878479294 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.147258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1191/2408] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:01:33.152284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.163360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fa18d-9c25-4c03-bd79-cfcedfe605bd admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.168191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.168846 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.169121 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.169393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] PATH_INFO: `/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4/users/024822935b5f4d8f899f87567e32b070/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.171444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae6dedc52f54fbd8d05195bacaa88a4, user_id=024822935b5f4d8f899f87567e32b070, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.174011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.185248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1218/2409] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:01:33 2022] PUT /identity/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4/users/024822935b5f4d8f899f87567e32b070/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:01:33.196356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915282a3-0240-4622-8787-66af77414ad0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.196666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915282a3-0240-4622-8787-66af77414ad0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.197115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915282a3-0240-4622-8787-66af77414ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.228665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-915282a3-0240-4622-8787-66af77414ad0 None None] MFA Rules not processed for user `024822935b5f4d8f899f87567e32b070`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:33.229899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-915282a3-0240-4622-8787-66af77414ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:33.303822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1192/2410] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:33.315876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.317419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:33.373041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:33.378892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8095c1-94ec-4042-9fc1-9edb35003ddd admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.383671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.384542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.385010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.385283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.394944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.396427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.406469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1219/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:33.490704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915282a3-0240-4622-8787-66af77414ad0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:33.504001 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915282a3-0240-4622-8787-66af77414ad0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:33.509549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:33.510207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:33.510486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:33.510751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:33.515216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:33.570968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:33.572539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:33.581569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1193/2412] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:36.895212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:36.896574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:36.974746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0bb86-62a8-4c9c-ad39-49919bb839f9 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:36.980445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:36.981060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:36.981372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:36.981637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:36.983386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:37.052797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:37.055372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e754-2177-4770-ac72-4754c444a4c4 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:37.065092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1220/2413] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:43.577837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=25 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:45.780788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:45.782327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:45.802114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68795c-ed1f-4174-954c-d04fa6f83cd9 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.808383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.809547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.809828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.810098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.812043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:45.825372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.827180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.837976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1194/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:47.060651 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=61 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:47.446705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baead9e-a284-4937-9579-d11396dcbce7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.446919 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baead9e-a284-4937-9579-d11396dcbce7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.447245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baead9e-a284-4937-9579-d11396dcbce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.469173 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7baead9e-a284-4937-9579-d11396dcbce7 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:47.470544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7baead9e-a284-4937-9579-d11396dcbce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.535788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1221/2415] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:47.549186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.550643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.645964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53136113-10e2-4fc3-930d-f9c07e28affb service nova] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.652233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.652931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.653285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.653633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.663633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.665410 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.678947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1195/2416] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:01:50.849182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.849994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.850417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.873795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] MFA Rules not processed for user `455284c947254207a719459efc031bea`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:50.875444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.948909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1222/2417] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:01:51.389325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:51.391358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.459800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:51.461468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.488096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888183f-5614-4158-ba21-4cc5f8241628 service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.495595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.496780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.497209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.497700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.500214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.514536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.516708 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.530835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1196/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:01:51.564378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d9a69-d7b8-4088-b7ff-cf24cabb689a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.568230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.568932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.571298 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.571623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.573932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:51.587857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.589465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f05cd-0b9d-4834-8741-408323819d9b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.602380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1223/2419] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:01:52.213458 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:52.214886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:52.350906 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca034e-78e5-494b-8a66-e041e0292432 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:52.357199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:52.358218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:52.358514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:52.359197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:52.362728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:52.389636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:52.391573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55591b48-30b8-476d-a2ad-2d94ca772862 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:52.403610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1197/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:01.596828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=76 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:02.398425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.383580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.384085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.384326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.386073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1224/2421] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 01:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:06.394647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231524-8b9d-4e1d-a903-7325211fafe2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.395163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231524-8b9d-4e1d-a903-7325211fafe2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.395310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231524-8b9d-4e1d-a903-7325211fafe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.416236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-fe231524-8b9d-4e1d-a903-7325211fafe2 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.417061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fe231524-8b9d-4e1d-a903-7325211fafe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.427130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1198/2422] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:06.442562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.444694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.516926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7056cb1-61c2-4c26-bf5a-56b294480e99 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.522616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.523370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.523720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.524092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.541370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.543123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ba22-686c-4c14-a344-58baa8add1de service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.559701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1225/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:06.652032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bfcb-a7bf-4e68-a95e-e7ba2d77e2b2 None None] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.652857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bfcb-a7bf-4e68-a95e-e7ba2d77e2b2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.653498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bfcb-a7bf-4e68-a95e-e7ba2d77e2b2 None None] PATH_INFO: `/` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.655979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1199/2424] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 01:02:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:06.661406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9af409-a704-41a9-9a40-7761e763441f None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.661783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9af409-a704-41a9-9a40-7761e763441f None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.662062 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9af409-a704-41a9-9a40-7761e763441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.685357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6b9af409-a704-41a9-9a40-7761e763441f None None] MFA Rules not processed for user `3acf83ba003e4e02a5ddd87c2714792d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.686911 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6b9af409-a704-41a9-9a40-7761e763441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.693977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1226/2425] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:06.715150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3bfcb-a7bf-4e68-a95e-e7ba2d77e2b2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.726661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3bfcb-a7bf-4e68-a95e-e7ba2d77e2b2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.730224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.730880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.731250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.731585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.739444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.741004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfef1c5-9528-4082-a827-38ffb4d41052 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.749803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1200/2426] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:06.867924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b04b7-4523-42d8-a203-33cecb276bb4 None None] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.868214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b04b7-4523-42d8-a203-33cecb276bb4 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.868490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b04b7-4523-42d8-a203-33cecb276bb4 None None] PATH_INFO: `/` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.869899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1227/2427] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 30 01:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:06.875331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009224-c423-45b6-8672-b5a00faff968 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.875616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009224-c423-45b6-8672-b5a00faff968 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.875885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18009224-c423-45b6-8672-b5a00faff968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.895639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-18009224-c423-45b6-8672-b5a00faff968 None None] MFA Rules not processed for user `28f8f65dc678403081189e31ef5f8a10`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.896826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-18009224-c423-45b6-8672-b5a00faff968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.906854 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1201/2428] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:06.917563 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b04b7-4523-42d8-a203-33cecb276bb4 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.930410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b04b7-4523-42d8-a203-33cecb276bb4 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.934615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.935268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.935530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.935802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.946573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.947987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f3378-47d8-476e-a211-97d3073464d2 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.958906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1228/2429] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:13.880141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18009224-c423-45b6-8672-b5a00faff968 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.958057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18009224-c423-45b6-8672-b5a00faff968 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.963344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.964426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.964426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.964642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] PATH_INFO: `/v3/users/6329687f545c42f7a60dbef46e1a74d3` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.966107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6329687f545c42f7a60dbef46e1a74d3)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.968232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.046104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6329687f545c42f7a60dbef46e1a74d3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.060038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6329687f545c42f7a60dbef46e1a74d3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.073654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6329687f545c42f7a60dbef46e1a74d3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.074615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6329687f545c42f7a60dbef46e1a74d3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.099545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6329687f545c42f7a60dbef46e1a74d3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:14.108751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1202/2430] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:13 2022] DELETE /identity/v3/users/6329687f545c42f7a60dbef46e1a74d3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.113547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56780f-9d69-4319-a7c9-7fca1da22d3c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.114162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56780f-9d69-4319-a7c9-7fca1da22d3c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.114162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56780f-9d69-4319-a7c9-7fca1da22d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.146142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4d56780f-9d69-4319-a7c9-7fca1da22d3c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.148635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4d56780f-9d69-4319-a7c9-7fca1da22d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.240927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1229/2431] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.245681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.247688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.330681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1c93-a014-4d5c-900a-917b0d4ee7ab admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.336268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.337133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.337963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.338598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.347620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.351589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.353336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.390806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1203/2432] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:14.598909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.599500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.599947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.623261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.625270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.732776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1230/2433] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:14.745748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.747388 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.896153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d9fb-5b3e-40d5-9dba-21881f0d2fd4 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.906865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.908948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.909446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.910362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.924969 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.927120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.939752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1204/2434] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:02:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:02:14.969066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.970748 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.017085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc42884-6066-4a07-b556-1d9578a13d0e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.047799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.048702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.048702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.050057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] PATH_INFO: `/v3/projects/571676f2f37b40288316568455af5f2b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.056234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=571676f2f37b40288316568455af5f2b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.058652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.137320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.179465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '571676f2f37b40288316568455af5f2b', 'request_id': 'req-ac20df01-45de-45e4-9f43-6c2ae4d33e92'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.188207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '571676f2f37b40288316568455af5f2b', 'request_id': 'req-ac20df01-45de-45e4-9f43-6c2ae4d33e92'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.195422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '571676f2f37b40288316568455af5f2b', 'request_id': 'req-ac20df01-45de-45e4-9f43-6c2ae4d33e92'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.228256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] The token cache is being invalidate because project 571676f2f37b40288316568455af5f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:15.229433 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.231531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.233417 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1231/2435] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:14 2022] DELETE /identity/v3/projects/571676f2f37b40288316568455af5f2b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:15.237398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.252476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c966c-5440-4f48-af93-e53fd0f1a075 service swift] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.383783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.384468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.384753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.385029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] PATH_INFO: `/v3/users/024822935b5f4d8f899f87567e32b070` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.386245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] RBAC: Authorizing `identity:delete_user(user_id=024822935b5f4d8f899f87567e32b070)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.388227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.491479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024822935b5f4d8f899f87567e32b070'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.513183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024822935b5f4d8f899f87567e32b070'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.527086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024822935b5f4d8f899f87567e32b070'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.528245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024822935b5f4d8f899f87567e32b070'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.557090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7beb8b5f-a468-4bef-a02d-de8e5a8d583b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024822935b5f4d8f899f87567e32b070'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:15.571361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1205/2436] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:15 2022] DELETE /identity/v3/users/024822935b5f4d8f899f87567e32b070 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:16.185057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.188355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.212061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20df01-45de-45e4-9f43-6c2ae4d33e92 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.334831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.335723 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.336061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.336394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] PATH_INFO: `/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.337766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae6dedc52f54fbd8d05195bacaa88a4)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.340227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.479896 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae6dedc52f54fbd8d05195bacaa88a4', 'request_id': 'req-0f00dc21-531b-4914-8891-736d87b8a861'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.490267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae6dedc52f54fbd8d05195bacaa88a4', 'request_id': 'req-0f00dc21-531b-4914-8891-736d87b8a861'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.505987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae6dedc52f54fbd8d05195bacaa88a4', 'request_id': 'req-0f00dc21-531b-4914-8891-736d87b8a861'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.546191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] The token cache is being invalidate because project 5ae6dedc52f54fbd8d05195bacaa88a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:16.547597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.552142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f00dc21-531b-4914-8891-736d87b8a861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:16.557525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1232/2437] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:16 2022] DELETE /identity/v3/projects/5ae6dedc52f54fbd8d05195bacaa88a4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:16.563453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.564476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.565006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.599841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.607901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:16.615901 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.737099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1206/2438] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:16.742828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.743136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.743410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.766098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:16.767659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:16.797060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1233/2439] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:16.798288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.823469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fa829-a579-4fc4-918e-4fc83be6b340 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.835778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.836636 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.836930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.837206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.837956 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:16.838408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.840228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.860879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1207/2440] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:16.864960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.876496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310fe9b-5304-4c93-9cff-48186feee9c3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.882096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.882787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.883098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.883371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.884022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.885964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.929283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1234/2441] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:16 2022] POST /identity/v3/projects => generated 357 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:16.935730 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.948731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927334bb-3cb4-4849-b1a8-31d7a889062b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.028692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.029640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.030358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.031204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.032284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.034683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.071711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1208/2442] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:16 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:17.076087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.092505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1bc58-ef72-4e91-8eed-d5d6a7ecd1b7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.098996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.099712 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.100036 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.100268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.100931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.103045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.118504 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1235/2443] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:17.119830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.134337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb8800-9296-4866-b115-b13fc9bd5fe0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.154789 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.156011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.156343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.156628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.158353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.159540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a378224-9597-41de-b9c1-38aea6d2afcf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.178376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.184915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1209/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:17.185383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.192031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf4cb5-f732-4fe1-9faa-37e8f407f0a2 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.200611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.201469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.201958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.202572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] PATH_INFO: `/v3/projects/e92a4f87fac5439ca7895877d016aa60/users/9641ee15b28c47f1950eecfe4eb30235/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.225444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92a4f87fac5439ca7895877d016aa60, user_id=9641ee15b28c47f1950eecfe4eb30235, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.228342 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.261692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.261988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.262253 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.266949 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1236/2445] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:17 2022] PUT /identity/v3/projects/e92a4f87fac5439ca7895877d016aa60/users/9641ee15b28c47f1950eecfe4eb30235/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:17.313205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] MFA Rules not processed for user `9641ee15b28c47f1950eecfe4eb30235`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.314644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.390665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.398572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1210/2446] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:17.399820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.484006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea780f8-2975-484c-bf8f-e51f9d8e24a9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.489505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.490406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.490685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.491045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.491684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.493613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.544897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1237/2447] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/projects => generated 357 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:17.546118 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.557143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3c12-0917-4463-99b2-00d84eea818d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.630915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.631653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.631934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.632115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.634530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.636693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.664570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1211/2448] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:17.668683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.684333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38dd77-59a0-4de8-9c49-1da5b2c4e81f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.691741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.692390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.692672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.692948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.693658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.695271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.707302 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1238/2449] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:17.711455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.728626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0ea58-5c6b-4e44-aad8-e732b01eaccf admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.737981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.738681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.738958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.739286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] PATH_INFO: `/v3/projects/18844752737f49648d04aebf0af6aa41/users/395195eadca146289e8f81446ae092cf/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.769031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18844752737f49648d04aebf0af6aa41, user_id=395195eadca146289e8f81446ae092cf, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.772642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.792456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1212/2450] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:17 2022] PUT /identity/v3/projects/18844752737f49648d04aebf0af6aa41/users/395195eadca146289e8f81446ae092cf/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:17.799256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.813555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ca05-eb19-4c0f-b294-edf02e932d16 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.874760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.875479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.875761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.876041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.876702 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.878479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a38cc1-cd70-4a77-9a21-b46f2799712a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.891228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1239/2451] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:17.894634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.907180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc2be7-b1eb-4489-89d2-7c7b9ff54fe1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.912013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.912698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.912978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.913255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] PATH_INFO: `/v3/projects/18844752737f49648d04aebf0af6aa41/users/395195eadca146289e8f81446ae092cf/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.915232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18844752737f49648d04aebf0af6aa41, user_id=395195eadca146289e8f81446ae092cf, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.917921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.932087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1213/2452] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:17 2022] PUT /identity/v3/projects/18844752737f49648d04aebf0af6aa41/users/395195eadca146289e8f81446ae092cf/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:17.944798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.945095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.945370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.976795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] MFA Rules not processed for user `395195eadca146289e8f81446ae092cf`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.978159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:18.021958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.068069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1240/2453] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:18.080021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.081833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:18.159773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c4704-3f73-4448-99ae-6ed9f44cefd5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.172434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.173082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.173359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.173631 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.193174 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.194835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25e520-2768-4347-9006-21a3765c5957 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.207342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1214/2454] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:18.327383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.348769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1ca27-88d9-4e05-84cf-1d1175179df7 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.361584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.362230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.362511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.362782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.465217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.466826 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.487181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1241/2455] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:27.849115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.849115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.849115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.869786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:27.871643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:27.934689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.944841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1215/2456] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 01:02:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:27.963406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.965187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.057334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b097f-9b77-488a-8ff2-1d47aadfd289 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.062946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.063617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.063895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.064167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.072945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.074573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.083875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1242/2457] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:02:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:02:29.931979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.933792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:29.991943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166901b-e4b1-4887-a340-958fb9fd7e2d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.995907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.996512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.996787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.997058 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.998809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.064166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.065612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.075927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1216/2458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:33.283462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.285372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.364791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.371695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f20cc3-1188-422b-875f-324e0e707c9b service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.374291 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.376273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.379136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.380030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.380398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.380758 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.383087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.392235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cb607-c3e1-48ec-a6db-961480f40776 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.404740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.405163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.405592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.405989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.406404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] PATH_INFO: `/v3/users/22f11e29c4ad4712a54877b0895d28a4` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.407898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.408048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f11e29c4ad4712a54877b0895d28a4)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.410068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.422352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1243/2459] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:33.477580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f11e29c4ad4712a54877b0895d28a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.485486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f11e29c4ad4712a54877b0895d28a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.493676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f11e29c4ad4712a54877b0895d28a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.494225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f11e29c4ad4712a54877b0895d28a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.506188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-88aff76b-68ac-40a9-8e57-270153ed0c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f11e29c4ad4712a54877b0895d28a4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.512146 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1217/2460] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:33 2022] DELETE /identity/v3/users/22f11e29c4ad4712a54877b0895d28a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:33.692978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.706160 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71079e-476c-4364-a594-89c043c87c8b service placement] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.777272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.777821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.778090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.778518 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] PATH_INFO: `/v3/projects/7e919c196d074b8989b4719267ce24f8` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.779695 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e919c196d074b8989b4719267ce24f8)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.781769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.831754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e919c196d074b8989b4719267ce24f8', 'request_id': 'req-d42c186c-3705-4283-bc40-fcdc1c94f01c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.838243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e919c196d074b8989b4719267ce24f8', 'request_id': 'req-d42c186c-3705-4283-bc40-fcdc1c94f01c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.844334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e919c196d074b8989b4719267ce24f8', 'request_id': 'req-d42c186c-3705-4283-bc40-fcdc1c94f01c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.858950 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] The token cache is being invalidate because project 7e919c196d074b8989b4719267ce24f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:33.859666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.859992 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d42c186c-3705-4283-bc40-fcdc1c94f01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:33.861505 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1244/2461] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:33 2022] DELETE /identity/v3/projects/7e919c196d074b8989b4719267ce24f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:33.866575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.866861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.867166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.887361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:33.888546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.907349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.956447 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1218/2462] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:33.964406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.964687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.964959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.986297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:33.987532 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.999396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1245/2463] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:34.005665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.016033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44221293-175e-4531-afb3-78e9a41ee5dd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.021177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.022004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.022128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.022310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.022963 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:34.023287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.024977 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.038063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1219/2464] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.042380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.052762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90426eec-2fea-4ff7-92b6-1c8bec878f28 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.059041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.059649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.059926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.060194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.060837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.063040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.095394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1246/2465] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:34.100126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.113974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371ee0c-7124-4c7e-911f-03a41a559c7f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.180694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.181307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.181579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.181851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.182846 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.184941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.212007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1220/2466] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:34.216558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.227959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd065a21-e912-48af-8913-5eb7a5212706 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.233753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.234522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.235377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.239076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.239778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.241400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.252641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1247/2467] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:34.256764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.267927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ecab7-af64-4693-971b-75167f101a2c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.275373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.275988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.276258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.276517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.277213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.278868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377db47-4dc1-4354-8a4b-b8944548fab3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.293676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1221/2468] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:34.294784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.305027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa0268-9056-4da7-b1d5-28913767d817 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.310474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.311486 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.311868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.312372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] PATH_INFO: `/v3/projects/fe19812dff73433b8b29d24ddbea3f56/users/9d7babb2d721414496d45840f9253852/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.328276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe19812dff73433b8b29d24ddbea3f56, user_id=9d7babb2d721414496d45840f9253852, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.331191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.346235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1248/2469] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:34 2022] PUT /identity/v3/projects/fe19812dff73433b8b29d24ddbea3f56/users/9d7babb2d721414496d45840f9253852/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:34.353252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.353545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.353822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.370612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.372360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.373748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.375632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.399467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.439269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1222/2470] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:34.443291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463a708-ff65-4496-aea0-4705678c6396 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.448171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.448868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.449004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.449170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.449452 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] PATH_INFO: `/v3/users/9641ee15b28c47f1950eecfe4eb30235` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.450422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.450775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9641ee15b28c47f1950eecfe4eb30235)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.452852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.486392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.506684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9641ee15b28c47f1950eecfe4eb30235'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.514127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9641ee15b28c47f1950eecfe4eb30235'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.514799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9641ee15b28c47f1950eecfe4eb30235'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.517911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e7d1e-c8d1-423a-9096-5f38d4f5d98c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.521898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.522508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.522786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.523086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.524994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.528249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9641ee15b28c47f1950eecfe4eb30235'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.538359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3a7f39c1-5746-4ae0-b8a4-4fefcd1099d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9641ee15b28c47f1950eecfe4eb30235'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:34.539992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.541567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.545277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1249/2471] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:34 2022] DELETE /identity/v3/users/9641ee15b28c47f1950eecfe4eb30235 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:34.550056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.550475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.550752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.552825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1223/2472] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:34.570559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.571840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.636386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1250/2473] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:34.652258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.653923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.665250 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdec45-174a-4bb1-98e0-a0656decad7f service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.719617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.719617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.719927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.720172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.730590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.732144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.743094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1224/2474] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:35.133656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.135284 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.152329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ea391-cd00-4ebe-9376-6745ced88c8e None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.157524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.158263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.158630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.159294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] PATH_INFO: `/v3/projects/e92a4f87fac5439ca7895877d016aa60` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.160161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92a4f87fac5439ca7895877d016aa60)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.163431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.204197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.219719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92a4f87fac5439ca7895877d016aa60', 'request_id': 'req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.227435 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92a4f87fac5439ca7895877d016aa60', 'request_id': 'req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.233613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92a4f87fac5439ca7895877d016aa60', 'request_id': 'req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.248455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] The token cache is being invalidate because project e92a4f87fac5439ca7895877d016aa60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:35.248843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.249539 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.251891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1251/2475] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:35 2022] DELETE /identity/v3/projects/e92a4f87fac5439ca7895877d016aa60 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:35.255375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.265546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aed41-75d4-46ac-a225-c71bbd99e3dc service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.348377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.349207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.349711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.350060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] PATH_INFO: `/v3/users/395195eadca146289e8f81446ae092cf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.351516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=395195eadca146289e8f81446ae092cf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.355688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.426975 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395195eadca146289e8f81446ae092cf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.436546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395195eadca146289e8f81446ae092cf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.447082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395195eadca146289e8f81446ae092cf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.448972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395195eadca146289e8f81446ae092cf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.461292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.464100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d4ecae43-495a-484c-bd47-10ba39ee9b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395195eadca146289e8f81446ae092cf'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:35.474419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1225/2476] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:35 2022] DELETE /identity/v3/users/395195eadca146289e8f81446ae092cf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:35.989634 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.991370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.005661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e0f27-cbe2-4cb5-ad3a-30bd364d6011 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.072130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.072929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.073057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.073324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] PATH_INFO: `/v3/projects/18844752737f49648d04aebf0af6aa41` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.074610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=18844752737f49648d04aebf0af6aa41)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.076620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.174055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18844752737f49648d04aebf0af6aa41', 'request_id': 'req-f013ac80-e8a0-4fbb-b426-154074e693bf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.182970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18844752737f49648d04aebf0af6aa41', 'request_id': 'req-f013ac80-e8a0-4fbb-b426-154074e693bf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.190611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18844752737f49648d04aebf0af6aa41', 'request_id': 'req-f013ac80-e8a0-4fbb-b426-154074e693bf'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.210076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] The token cache is being invalidate because project 18844752737f49648d04aebf0af6aa41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:36.210370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.211092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f013ac80-e8a0-4fbb-b426-154074e693bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:36.212622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1252/2477] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:35 2022] DELETE /identity/v3/projects/18844752737f49648d04aebf0af6aa41 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:36.218010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.218399 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.218677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.239756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.240988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.324455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1226/2478] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:36.333114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.333393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.333677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.353775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.355301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.363627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1253/2479] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:36.368234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.378455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee694f-83d2-40c3-9e72-9787a2dfa3bb None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.384300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.384894 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.385254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.385444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.386092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:36.386389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.388029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.401180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1227/2480] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:36.405828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.416272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3508c-f8f7-46fa-aee8-4a021004a7d5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.421296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.422362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.422642 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.422930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.423607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.425533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.460181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1254/2481] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:36.464877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.476701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9177c0-8422-487a-a2d2-7b5c3d9a7fa0 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.539102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.539713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.539990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.540260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.541259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.543344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.567338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1228/2482] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:36.571677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.582972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b935-2fa8-4a6e-9be4-f571a1a41686 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.588401 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.589027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.589306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.589574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.590238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.592026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.604117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1255/2483] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:36.608182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.619879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f6d27-1011-40a5-a389-6bb11e84d408 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.625712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.626363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.626639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.626925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.627616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.629334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2163a55-582d-4291-8172-656e7f71222e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.641125 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1229/2484] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:36.648190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.661384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc54a57-912f-4577-9f2c-c3dd6eed9ba9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.666239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.666931 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.667247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.667520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] PATH_INFO: `/v3/projects/aa8c8fe185bc4161bbf29159eff56dd9/users/eda32a6b6350484e9eeda03c41a8dd62/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.677683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.686468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8c8fe185bc4161bbf29159eff56dd9, user_id=eda32a6b6350484e9eeda03c41a8dd62, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.690170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.715360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1256/2485] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:36 2022] PUT /identity/v3/projects/aa8c8fe185bc4161bbf29159eff56dd9/users/eda32a6b6350484e9eeda03c41a8dd62/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:36.718212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.718614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.718892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.749437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] MFA Rules not processed for user `eda32a6b6350484e9eeda03c41a8dd62`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.752154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.806199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.831610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1230/2486] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:36.836123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.912851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366de7b-861b-4105-a0ca-f9c9749d7e91 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:36.917771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:36.918415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.918692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.918968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.919701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:36.921918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:36.959277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1257/2487] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/projects => generated 401 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:36.959788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.973656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc730c2b-0dc8-4b97-9ecc-c37ad4c819b2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.045177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.045520 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.045841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.046153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.047101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.049087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.085369 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1231/2488] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/users => generated 307 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:37.085595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.095825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5102-daa1-4a51-967b-2f809d2ecee9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.100550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.101425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.101579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.101894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.102744 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.104795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.120394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1258/2489] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:37.121661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.137816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802e445-cc65-4a0e-9edd-b6a639bb0f75 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.143578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.144294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.144571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.144843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] PATH_INFO: `/v3/projects/d71ca0b43dcf4121868d76e81cd0127b/users/9371583d86424dd6a64837603a18264e/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.165697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71ca0b43dcf4121868d76e81cd0127b, user_id=9371583d86424dd6a64837603a18264e, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.168344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.193247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1232/2490] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:37 2022] PUT /identity/v3/projects/d71ca0b43dcf4121868d76e81cd0127b/users/9371583d86424dd6a64837603a18264e/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:37.194614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.205724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc72fb-3f24-411d-b10c-7c22e9ba62d9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.292841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.293479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.301560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.301560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.301560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.303139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a8d2c-8223-4149-a7de-a83fb693b09c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.322716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1259/2491] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:37.326771 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.338959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58aa35-4dad-472a-9f55-92b23fd10b2f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.347795 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.348499 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.348779 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.349053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] PATH_INFO: `/v3/projects/d71ca0b43dcf4121868d76e81cd0127b/users/9371583d86424dd6a64837603a18264e/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.351396 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71ca0b43dcf4121868d76e81cd0127b, user_id=9371583d86424dd6a64837603a18264e, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.354079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.366510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1233/2492] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:37 2022] PUT /identity/v3/projects/d71ca0b43dcf4121868d76e81cd0127b/users/9371583d86424dd6a64837603a18264e/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:37.378479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.378479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.378755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.418066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] MFA Rules not processed for user `9371583d86424dd6a64837603a18264e`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.419293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.487548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.531426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1260/2493] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:37.542015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.543766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.640256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b586a9-f834-41e1-abb1-200e7fca0ebd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.649671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.650803 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.651436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.651995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.672580 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.675984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041e9f7-04cc-4d81-b150-819ebc70e710 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.711188 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1234/2494] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:37.840194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.853948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e6df-8dfd-4e6a-b42b-d0e8643b4d75 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.857592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.859258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.859258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.859258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.932035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.934944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.946456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1261/2495] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:39.350529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a9ebd-7bbb-468e-9270-107a3b687c3f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.351255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a9ebd-7bbb-468e-9270-107a3b687c3f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.351255 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a9ebd-7bbb-468e-9270-107a3b687c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.382185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d32a9ebd-7bbb-468e-9270-107a3b687c3f None None] MFA Rules not processed for user `9d7babb2d721414496d45840f9253852`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:39.383565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d32a9ebd-7bbb-468e-9270-107a3b687c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.420586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:39.456400 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1235/2496] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:39.471431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.473347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.568387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da799d-72c7-49ef-a82c-5f4e01ae2930 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.573690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.574498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.574789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.575381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.587510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.589290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.603611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1262/2497] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:40.954476 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.954995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.954995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.974858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] MFA Rules not processed for user `9d7babb2d721414496d45840f9253852`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:40.975867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.985453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1236/2498] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:41.336117 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.338465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.403033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548523b-61a4-4124-93f6-f44d71fc93d5 service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.407199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.407816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.408094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.408924 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.411263 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.423717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.425220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7077b-6744-4ac5-9641-2edc1fe9b446 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.435550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1263/2499] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:42.451603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:42.528735 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be42f-4475-4767-b6af-1586a3d50c6c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.536812 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.537807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.538564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.538840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] PATH_INFO: `/v3/users/eda32a6b6350484e9eeda03c41a8dd62` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.540780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda32a6b6350484e9eeda03c41a8dd62)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.542769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.596279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda32a6b6350484e9eeda03c41a8dd62'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.605060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda32a6b6350484e9eeda03c41a8dd62'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.615699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda32a6b6350484e9eeda03c41a8dd62'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.616347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda32a6b6350484e9eeda03c41a8dd62'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.630996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda32a6b6350484e9eeda03c41a8dd62'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:42.637147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1237/2500] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:42 2022] DELETE /identity/v3/users/eda32a6b6350484e9eeda03c41a8dd62 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:42.641942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.642479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.642479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.663314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:42.664591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.668848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:42.729156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1264/2501] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:42.739181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:42.740756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.798323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:42.804516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea95e0-4ffb-41e0-9376-2e467bf59894 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.811332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.812257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.812551 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.813172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.827503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.829074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.839845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1238/2502] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:43.180366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.181974 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.199698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8eea2-5a38-468a-8939-be1d88b6628c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.208054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.208818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.209326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.209819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] PATH_INFO: `/v3/projects/aa8c8fe185bc4161bbf29159eff56dd9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.211324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8c8fe185bc4161bbf29159eff56dd9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.227718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:43.281892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8c8fe185bc4161bbf29159eff56dd9', 'request_id': 'req-57c850ea-4990-4c1c-9387-5876861e3bee'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.288432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8c8fe185bc4161bbf29159eff56dd9', 'request_id': 'req-57c850ea-4990-4c1c-9387-5876861e3bee'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.294283 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8c8fe185bc4161bbf29159eff56dd9', 'request_id': 'req-57c850ea-4990-4c1c-9387-5876861e3bee'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.311212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] The token cache is being invalidate because project aa8c8fe185bc4161bbf29159eff56dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:43.311617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.312456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.314276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1265/2503] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:43 2022] DELETE /identity/v3/projects/aa8c8fe185bc4161bbf29159eff56dd9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:43.318314 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.328559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f6110-913f-40cc-8e37-9b0aa9483987 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:43.395165 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:43.395814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.396285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.396412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] PATH_INFO: `/v3/users/9371583d86424dd6a64837603a18264e` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.397635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9371583d86424dd6a64837603a18264e)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:43.399461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:43.448829 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9371583d86424dd6a64837603a18264e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.456616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9371583d86424dd6a64837603a18264e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.465533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9371583d86424dd6a64837603a18264e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.466286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9371583d86424dd6a64837603a18264e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.479360 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-d85a60d8-44b6-4299-9a24-16ea1c6a7113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9371583d86424dd6a64837603a18264e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:43.485653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1239/2504] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:43 2022] DELETE /identity/v3/users/9371583d86424dd6a64837603a18264e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:43.945798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:43.947468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.968831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c850ea-4990-4c1c-9387-5876861e3bee admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.078171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.079305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.079794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.080291 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] PATH_INFO: `/v3/projects/d71ca0b43dcf4121868d76e81cd0127b` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.081707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71ca0b43dcf4121868d76e81cd0127b)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.085035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.139067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:44.152925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71ca0b43dcf4121868d76e81cd0127b', 'request_id': 'req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:44.160011 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71ca0b43dcf4121868d76e81cd0127b', 'request_id': 'req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:44.166139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71ca0b43dcf4121868d76e81cd0127b', 'request_id': 'req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:44.183330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] The token cache is being invalidate because project d71ca0b43dcf4121868d76e81cd0127b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:44.183623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:44.184296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0c8be8cb-b34b-4e4a-915e-9cfce20cc30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:44.186187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1266/2505] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:43 2022] DELETE /identity/v3/projects/d71ca0b43dcf4121868d76e81cd0127b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:44.192180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.192681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.193190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.214583 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.215792 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.279808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1240/2506] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:44.287958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.288245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.288522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.308998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.310129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.319340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1267/2507] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:44.323767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.334457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbc643-5d28-42f1-8a9e-08093883931e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.339952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.340591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.340874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.341144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.341790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:44.342106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.343776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.346964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:44.361911 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1241/2508] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:44.363140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.372932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebafe3-a107-4189-b486-20c8c323f65c None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.378064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.378756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.379072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.379344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.379985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.381888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.414412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1268/2509] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:44.418375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.429204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267985bb-10b5-4ec7-974a-e24ab29d45b9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.487558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.488258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.488437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.488710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.489794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.491293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.514434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1242/2510] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:44.519243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.530324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146cec1-2162-472d-a609-aaa96e0ba1b8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.535382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.536070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.536409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.536865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.537589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.539304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.551656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1269/2511] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:44.555635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.566888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7fc32-e3bd-42e4-b873-08e9372d030b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.571940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.572659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.572953 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.573197 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.573924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.575654 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f552e-0678-4485-b214-66a2dc29162e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.587441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1243/2512] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:44.591952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.602577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31714033-6f5a-4348-8c03-7dbfd89e1952 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.607862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.608503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.608819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.609086 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] PATH_INFO: `/v3/projects/1613a632a1b7470b93d77309bebcd1e6/users/05fed2ce580f49609b87902c0c637d96/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.626307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1613a632a1b7470b93d77309bebcd1e6, user_id=05fed2ce580f49609b87902c0c637d96, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.629784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.647109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1270/2513] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:44 2022] PUT /identity/v3/projects/1613a632a1b7470b93d77309bebcd1e6/users/05fed2ce580f49609b87902c0c637d96/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:44.722340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.722616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.722883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.751395 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] MFA Rules not processed for user `05fed2ce580f49609b87902c0c637d96`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.752612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.820419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1244/2514] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:44.824659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.888538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a43a1-9b45-412b-b41b-16c58e741e7c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.893772 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.894525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.894671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.894906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.895566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:44.897645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:44.927178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1271/2515] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:44.931451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.942763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023748a-c848-4756-bf2b-17d531c3408d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.008814 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.009504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.009823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.010064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.010718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.012780 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.025594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.042569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1245/2516] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:45.043664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.055241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfb2e2-74a9-4deb-8313-9a4ded4b3c8f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.062358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.063072 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.063292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.063557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.064205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.065824 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.077340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1272/2517] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:45.083826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.094857 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb8c11-b592-4509-9997-6d4c74b3b53e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.105271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.106350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.106625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.106895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] PATH_INFO: `/v3/projects/be5142e951784ba789e1a113b04c7e1a/users/91eceda10d9c4c2d80f25df4dc06e530/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.127211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5142e951784ba789e1a113b04c7e1a, user_id=91eceda10d9c4c2d80f25df4dc06e530, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.131610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.149157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1246/2518] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:45 2022] PUT /identity/v3/projects/be5142e951784ba789e1a113b04c7e1a/users/91eceda10d9c4c2d80f25df4dc06e530/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:45.153300 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.163860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f5d3-780c-47b2-86ce-7b74d69ba58f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.197010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.228410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.229038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.229311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.229571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.230215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.232003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07693623-034b-456d-8ed0-d37c6df09e90 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.242481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1273/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:45.247048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.257869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6986825-2a0b-44c8-9f6e-76c925a6077a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.263186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.263915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.264272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.264621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] PATH_INFO: `/v3/projects/be5142e951784ba789e1a113b04c7e1a/users/91eceda10d9c4c2d80f25df4dc06e530/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.267101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5142e951784ba789e1a113b04c7e1a, user_id=91eceda10d9c4c2d80f25df4dc06e530, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.269831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.281513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1247/2520] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:45 2022] PUT /identity/v3/projects/be5142e951784ba789e1a113b04c7e1a/users/91eceda10d9c4c2d80f25df4dc06e530/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:45.292035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.292321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.292591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.323756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] MFA Rules not processed for user `91eceda10d9c4c2d80f25df4dc06e530`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:45.325135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.409530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1274/2521] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:45.421327 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.423294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.486523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f1941-743d-4cc0-a01c-794f7acbcb18 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.490542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.491228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.491480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.491756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.501147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.502588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aaf44-bfb8-47b1-b4a7-bdffca21edbc service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.511989 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1248/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:45.651810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.716672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323212f4-c531-45b5-b81b-b0edd0faece5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.721528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.722170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.722441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.722731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] PATH_INFO: `/v3/users/05fed2ce580f49609b87902c0c637d96` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.724019 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05fed2ce580f49609b87902c0c637d96)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.725947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.776167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05fed2ce580f49609b87902c0c637d96'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:45.784185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05fed2ce580f49609b87902c0c637d96'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:45.785365 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05fed2ce580f49609b87902c0c637d96'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:45.787081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.803099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05fed2ce580f49609b87902c0c637d96'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:45.812244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05fed2ce580f49609b87902c0c637d96'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:45.818497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1275/2523] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:45 2022] DELETE /identity/v3/users/05fed2ce580f49609b87902c0c637d96 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:45.823106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10794e06-34ab-4641-9f98-157304f91091 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.823438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10794e06-34ab-4641-9f98-157304f91091 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.823719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10794e06-34ab-4641-9f98-157304f91091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.843525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-10794e06-34ab-4641-9f98-157304f91091 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:45.844806 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-10794e06-34ab-4641-9f98-157304f91091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.909286 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1249/2524] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:45.921066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.922844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.984721 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adec074-caf6-468c-a30a-3c84faddcc25 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.989592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.990193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.990583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.990892 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.001344 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.002796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.012326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1276/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:46.379753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10794e06-34ab-4641-9f98-157304f91091 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.381435 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-10794e06-34ab-4641-9f98-157304f91091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:46.400892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10794e06-34ab-4641-9f98-157304f91091 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.416923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.417650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.417934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.418208 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] PATH_INFO: `/v3/projects/1613a632a1b7470b93d77309bebcd1e6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.419424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1613a632a1b7470b93d77309bebcd1e6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.421564 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.507193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1613a632a1b7470b93d77309bebcd1e6', 'request_id': 'req-4f2df06f-9819-4acc-8d94-2516b4b323b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.512973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.519075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1613a632a1b7470b93d77309bebcd1e6', 'request_id': 'req-4f2df06f-9819-4acc-8d94-2516b4b323b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.533627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1613a632a1b7470b93d77309bebcd1e6', 'request_id': 'req-4f2df06f-9819-4acc-8d94-2516b4b323b7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.546951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] The token cache is being invalidate because project 1613a632a1b7470b93d77309bebcd1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:46.547275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.547934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.549436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1250/2526] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:46 2022] DELETE /identity/v3/projects/1613a632a1b7470b93d77309bebcd1e6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:46.553589 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.564478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a76d-b466-4f0a-a2c2-671b74936588 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.629119 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.629848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.630183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.630516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] PATH_INFO: `/v3/users/91eceda10d9c4c2d80f25df4dc06e530` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.631751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91eceda10d9c4c2d80f25df4dc06e530)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.633755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.685101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91eceda10d9c4c2d80f25df4dc06e530'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.692379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91eceda10d9c4c2d80f25df4dc06e530'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.693233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91eceda10d9c4c2d80f25df4dc06e530'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.705501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91eceda10d9c4c2d80f25df4dc06e530'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.709131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.714438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e367626-c103-4daf-9b51-efc796379a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91eceda10d9c4c2d80f25df4dc06e530'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:46.722917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1277/2527] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:46 2022] DELETE /identity/v3/users/91eceda10d9c4c2d80f25df4dc06e530 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:47.026063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.027753 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:47.041217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2df06f-9819-4acc-8d94-2516b4b323b7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.099267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.099907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.100185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.100456 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] PATH_INFO: `/v3/projects/be5142e951784ba789e1a113b04c7e1a` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.101698 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5142e951784ba789e1a113b04c7e1a)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.103670 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.155273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5142e951784ba789e1a113b04c7e1a', 'request_id': 'req-1ed845b3-592c-4828-8674-fbd0f8ab786f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:47.160830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5142e951784ba789e1a113b04c7e1a', 'request_id': 'req-1ed845b3-592c-4828-8674-fbd0f8ab786f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:47.168980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5142e951784ba789e1a113b04c7e1a', 'request_id': 'req-1ed845b3-592c-4828-8674-fbd0f8ab786f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:47.180304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] The token cache is being invalidate because project be5142e951784ba789e1a113b04c7e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:47.180585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:47.181242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-1ed845b3-592c-4828-8674-fbd0f8ab786f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:47.183144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1251/2528] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:47 2022] DELETE /identity/v3/projects/be5142e951784ba789e1a113b04c7e1a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:47.188183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.188468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.188738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.208980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:47.210107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:47.283793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1278/2529] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:47.291776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.292063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.292342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.314863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:47.316116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:47.326065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1252/2530] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:47.330829 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.343101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93b65a-b879-4937-b426-736f8d1fd922 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.351672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.352482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.352840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.353190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.354177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:47.354604 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.357181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.370157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1279/2531] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:47.375039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.384651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1db27-06cf-49d8-9839-d390b5d1168b None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.390189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.390878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.391241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.391459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.397027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.397027 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.428964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1253/2532] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:47.433701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.444374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fce91-3544-48b2-8ab2-51ef176102a2 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.505833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.506443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.506716 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.506988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.507662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.509635 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.532142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1280/2533] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:47.536619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.547571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd27e6-856e-41d1-8eba-e0ce48f199f7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.552688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.553424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.553553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.553825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.554461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.556126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.567495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1254/2534] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:47.571549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.583508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e97e5a-86ca-4a8c-8ffc-24708b24886c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.590053 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.590921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.591322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.591688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.592686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.594805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929350b-dd1b-4fc0-8042-e6b5aac06c6b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.606030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1281/2535] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:47.610503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.621000 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbb6f-0586-4eb7-b9ac-233a724e89fa admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.627279 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.628196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.628484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.628759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] PATH_INFO: `/v3/projects/2cc1e0e74c2f403595c4ba7a1b05a691/users/26a04c849c4f49a693ac7f48ac5f00c9/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.645817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc1e0e74c2f403595c4ba7a1b05a691, user_id=26a04c849c4f49a693ac7f48ac5f00c9, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.648525 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.665688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1255/2536] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:47 2022] PUT /identity/v3/projects/2cc1e0e74c2f403595c4ba7a1b05a691/users/26a04c849c4f49a693ac7f48ac5f00c9/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:47.677522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.677811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.678103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.709324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] MFA Rules not processed for user `26a04c849c4f49a693ac7f48ac5f00c9`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:47.710550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:47.762724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.781157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1282/2537] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:47.786070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.796553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.851782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35454f2e-fdab-446e-901c-4179525964fd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.860668 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.861325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.861601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.862486 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.863181 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.865841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.895873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1256/2538] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:47.900295 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.919491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaaab3-592f-424d-9092-2bc0338b9325 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.990201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.990743 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.991038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.991363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.992031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.994030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.017582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1283/2539] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:48.021811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.033840 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1e2a-6c79-47fb-ad98-1622128dc3e9 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.039317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.039950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.040229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.040556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.041258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.042880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.054478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1257/2540] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:48.059473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.070879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602913b-a4f4-4674-9c3a-4a0beb575dfc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.076816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.077464 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.077752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.078172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] PATH_INFO: `/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f/users/d91191d9050e4540bddf0edcdf031cc1/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.095696 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4b661cadd9467aa3ef213bb1afa64f, user_id=d91191d9050e4540bddf0edcdf031cc1, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.098448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.114874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1284/2541] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:48 2022] PUT /identity/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f/users/d91191d9050e4540bddf0edcdf031cc1/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:48.120283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.132659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bbb56-664c-4ae7-b6d2-94d7612d2f1b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.199311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.199933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.200211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.200484 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.201120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.202737 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d278f6-0457-4245-8a87-aab624cd6fec admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.213751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1258/2542] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:48.218317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.229054 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bdf1-6c84-4c77-83a9-93bfcdc42462 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.234388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.235715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.236023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.236275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] PATH_INFO: `/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f/users/d91191d9050e4540bddf0edcdf031cc1/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.238257 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4b661cadd9467aa3ef213bb1afa64f, user_id=d91191d9050e4540bddf0edcdf031cc1, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.241016 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.254754 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1285/2543] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:48 2022] PUT /identity/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f/users/d91191d9050e4540bddf0edcdf031cc1/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:48.265325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.265816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.266212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.295412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] MFA Rules not processed for user `d91191d9050e4540bddf0edcdf031cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:48.296747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.372149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1259/2544] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:48.384111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.386233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.466166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42a6a-2fd4-4e49-bdc4-fb36799bb940 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.470382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.471040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.471322 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.471593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.482218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.483624 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536c76e-329d-4268-a930-606bf15200bc service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.494138 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1286/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:48.925227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.931190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.994141 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d48b9-8ba9-41a9-a3fe-4ae992981df5 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.998889 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.999566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.999843 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.000225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] PATH_INFO: `/v3/users/26a04c849c4f49a693ac7f48ac5f00c9` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.001315 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a04c849c4f49a693ac7f48ac5f00c9)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.003325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.053014 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a04c849c4f49a693ac7f48ac5f00c9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.060281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a04c849c4f49a693ac7f48ac5f00c9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.070207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a04c849c4f49a693ac7f48ac5f00c9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.071105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a04c849c4f49a693ac7f48ac5f00c9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.083358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a04c849c4f49a693ac7f48ac5f00c9'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.089672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1260/2546] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:48 2022] DELETE /identity/v3/users/26a04c849c4f49a693ac7f48ac5f00c9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:49.094366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.094880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.094981 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.114282 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:49.115524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:49.157156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.178585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1287/2547] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:49.190993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.192632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:49.252205 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25adac-50b8-49ae-b7d4-d7d8163cfa89 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.256528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.257132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.257405 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.257671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.268210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.269666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.279512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1261/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:49.564811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.566274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:49.578917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb53a9-50a6-4254-944c-344c672ae229 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.584393 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.585046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.585324 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.585614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] PATH_INFO: `/v3/projects/2cc1e0e74c2f403595c4ba7a1b05a691` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.586865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc1e0e74c2f403595c4ba7a1b05a691)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.588818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.636211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc1e0e74c2f403595c4ba7a1b05a691', 'request_id': 'req-cc49e18f-3e8b-4d64-a612-23d899236f40'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.642383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc1e0e74c2f403595c4ba7a1b05a691', 'request_id': 'req-cc49e18f-3e8b-4d64-a612-23d899236f40'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.647996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc1e0e74c2f403595c4ba7a1b05a691', 'request_id': 'req-cc49e18f-3e8b-4d64-a612-23d899236f40'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.663354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] The token cache is being invalidate because project 2cc1e0e74c2f403595c4ba7a1b05a691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:49.663646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.664331 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.665891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1288/2549] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:49 2022] DELETE /identity/v3/projects/2cc1e0e74c2f403595c4ba7a1b05a691 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:49.672066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.682724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1c130-39e1-42ff-8381-434622b9d27a service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.706643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.741145 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.741784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.742062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.742333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] PATH_INFO: `/v3/users/d91191d9050e4540bddf0edcdf031cc1` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.743504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91191d9050e4540bddf0edcdf031cc1)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.745438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.799089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91191d9050e4540bddf0edcdf031cc1'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.809659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91191d9050e4540bddf0edcdf031cc1'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.820532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91191d9050e4540bddf0edcdf031cc1'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.821296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91191d9050e4540bddf0edcdf031cc1'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.834046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-bca2077c-3f34-49d7-883a-56f81395e283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91191d9050e4540bddf0edcdf031cc1'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:49.840169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1262/2550] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:49 2022] DELETE /identity/v3/users/d91191d9050e4540bddf0edcdf031cc1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:50.127804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.129858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.151048 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49e18f-3e8b-4d64-a612-23d899236f40 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.209880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.210520 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.210797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.211097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] PATH_INFO: `/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.212233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4b661cadd9467aa3ef213bb1afa64f)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.214156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.274456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4b661cadd9467aa3ef213bb1afa64f', 'request_id': 'req-0f1ce8eb-60df-4113-a276-08fed7dd4768'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:50.283823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4b661cadd9467aa3ef213bb1afa64f', 'request_id': 'req-0f1ce8eb-60df-4113-a276-08fed7dd4768'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:50.292814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4b661cadd9467aa3ef213bb1afa64f', 'request_id': 'req-0f1ce8eb-60df-4113-a276-08fed7dd4768'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:50.306555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.311820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] The token cache is being invalidate because project 6b4b661cadd9467aa3ef213bb1afa64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:50.312164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:50.313194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0f1ce8eb-60df-4113-a276-08fed7dd4768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:50.315220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1289/2551] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:50 2022] DELETE /identity/v3/projects/6b4b661cadd9467aa3ef213bb1afa64f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:50.320364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.320867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.320976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.341554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:50.342685 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.406736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1263/2552] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:50.416177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.416525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.416786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.439109 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:50.440360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.448971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1290/2553] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:50.453478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.463398 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ebbff-25ef-4d88-8b55-ed9518ee9037 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.468050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.468669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.468944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.469193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.469915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:50.470193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.471860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.484175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1264/2554] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:50.488585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.498299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb5373-e6f9-41db-82ca-66ffa87218ec None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.504115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.504722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.504999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.505271 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.505918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.507887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.540205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1291/2555] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:50.544676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.555052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd01dc-763d-4561-b1df-088014d7de2f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.587330 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.616717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.617353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.617665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.617935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.618556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.620512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.644472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1265/2556] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:50.648852 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.659775 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebd70b-0e72-413d-b62f-6d75e2618bc8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.664917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.665530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.665855 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.666122 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.666790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.668458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.678692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1292/2557] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:50.683196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.693773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd6bef-771d-4305-aa0e-06fdcd275bcf admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.698937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.699566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.699841 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.700107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.700739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.702328 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a0292-cff0-4122-8a27-5c45ec92ef44 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.712584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1266/2558] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:50.716857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.727735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8f97-b179-4536-875a-eea831427be4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.733508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.734161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.734462 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.734879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] PATH_INFO: `/v3/projects/2515b35c728446fdbf64358f24b1d1f3/users/bcaf4c3dfcc248b89e1147419f7cc440/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.750662 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2515b35c728446fdbf64358f24b1d1f3, user_id=bcaf4c3dfcc248b89e1147419f7cc440, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.753397 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.768472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1293/2559] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:50 2022] PUT /identity/v3/projects/2515b35c728446fdbf64358f24b1d1f3/users/bcaf4c3dfcc248b89e1147419f7cc440/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:50.779866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.780241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.780628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.811252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] MFA Rules not processed for user `bcaf4c3dfcc248b89e1147419f7cc440`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:50.812495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.878674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1267/2560] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:50.883491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.952661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48394d-ca80-4ecf-b7f6-08b5ccfedc64 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.958170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.958778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.959049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.959350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.960007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.961921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.991133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1294/2561] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:50.995992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.012430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06700ba-9b6f-4aff-a454-5178be9a39a2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.074045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.074578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.074858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.075161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.075818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.077761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.102090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1268/2562] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:51.106171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.117654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484012-5321-4046-8151-f8842c56fc3b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.122684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.123327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.123636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.123937 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.124587 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.126475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.137836 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1295/2563] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:51.142585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.152769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1712ba-b3c4-482c-87b2-b095b46491d5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.157666 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.158277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.158554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.158819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.159534 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.161170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081def88-f29f-4e90-80f1-11a328239b1a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.174433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1269/2564] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:51.178606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.190497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac6c98-2f6b-4d13-8108-55e5002a494b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.195930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.196735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.197080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.197420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] PATH_INFO: `/v3/projects/f43e77776607462fb2dfd746d7bca3c9/users/d75475583a2c451a85dd90de1615f4c4/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.214023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43e77776607462fb2dfd746d7bca3c9, user_id=d75475583a2c451a85dd90de1615f4c4, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.217311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.243501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1296/2565] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:51 2022] PUT /identity/v3/projects/f43e77776607462fb2dfd746d7bca3c9/users/d75475583a2c451a85dd90de1615f4c4/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:51.245109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.245657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.245996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.277942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] MFA Rules not processed for user `d75475583a2c451a85dd90de1615f4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:51.279252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:51.288391 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:51.369613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1270/2566] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:51.373548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.435388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:51.454068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503d77-de60-449d-b7bb-75433251a548 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.459626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.460523 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.460925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.461285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.462027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.464094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.500338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1297/2567] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:02:51 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:51.504955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.517070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d59b3-08ad-4312-82a6-37b71a82d23a None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.589465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.589993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.590268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.590536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.591200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.594915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.622028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1271/2568] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:02:51 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:51.626227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.637638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea764550-b5c1-4e12-980c-018f1cb5709c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.643715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.644458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.644832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.645586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.645934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.647659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.662085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1298/2569] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:02:51.666619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.677830 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce112c-4b93-46f4-8839-7b81a1d324ae admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.683288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.683965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.684244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.684518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] PATH_INFO: `/v3/projects/67ad5f47152b4544bc863f8b75b43825/users/d5310603cbde423e9c2bb838cdd24915/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.700776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ad5f47152b4544bc863f8b75b43825, user_id=d5310603cbde423e9c2bb838cdd24915, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.704106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.718893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1272/2570] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:51 2022] PUT /identity/v3/projects/67ad5f47152b4544bc863f8b75b43825/users/d5310603cbde423e9c2bb838cdd24915/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:51.723184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.733847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fdc24-63fb-469c-94f6-bc54a52616df admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.793554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.794205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.794478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.794752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.795453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.797316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2beffc8-4f52-4277-a4dd-c18b8c48cdb5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.808255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1299/2571] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:02:51.812672 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.843496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d182-8ae9-4746-a0ef-1a2f620c206b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.850628 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.851380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.851637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.851935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] PATH_INFO: `/v3/projects/67ad5f47152b4544bc863f8b75b43825/users/d5310603cbde423e9c2bb838cdd24915/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.854000 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ad5f47152b4544bc863f8b75b43825, user_id=d5310603cbde423e9c2bb838cdd24915, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.856682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.869231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1273/2572] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:02:51 2022] PUT /identity/v3/projects/67ad5f47152b4544bc863f8b75b43825/users/d5310603cbde423e9c2bb838cdd24915/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:51.882535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.882930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.883314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.921211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] MFA Rules not processed for user `d5310603cbde423e9c2bb838cdd24915`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:51.922699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.000965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1300/2573] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:52.014034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:52.015902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.075939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2750d4-0f32-483c-af63-8c76ef8bb9ba admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:52.079675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:52.080285 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.080584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.080852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.094433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:52.152010 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:52.153487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:52.165022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1274/2574] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:52.520585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:52.522044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.576831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:52.633759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51aa1-5293-4d91-aa4c-af3655dda8ec None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:52.655804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:52.656693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.657051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.657454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.662468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.685279 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:52.687225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:52.701075 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1301/2575] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:53.909995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.911595 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:53.924316 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed1e1b-dde9-430c-a750-f000e2fb6a87 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.928179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.928776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.929052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.929317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.991983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:53.993342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.002893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1275/2576] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:54.724960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.735426 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09825ac-2c2f-4af0-8159-e9707b462f60 service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.740022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.740627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.740908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.741181 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.752471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.754713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b184535-ce78-46ed-a506-29b1959da3c9 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.768731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1302/2577] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:58.731936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.798299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db601570-a796-4353-bb78-bf275d5d470f service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:58.803794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:58.804426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.804694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.804957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] PATH_INFO: `/v3/users/bcaf4c3dfcc248b89e1147419f7cc440` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.806182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcaf4c3dfcc248b89e1147419f7cc440)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:58.808503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:58.860269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcaf4c3dfcc248b89e1147419f7cc440'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:58.867342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcaf4c3dfcc248b89e1147419f7cc440'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:58.869093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:58.876683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcaf4c3dfcc248b89e1147419f7cc440'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:58.878075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcaf4c3dfcc248b89e1147419f7cc440'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:58.891273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcaf4c3dfcc248b89e1147419f7cc440'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:58.897893 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1276/2578] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:58 2022] DELETE /identity/v3/users/bcaf4c3dfcc248b89e1147419f7cc440 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:58.903644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.904170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.904278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.925927 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:58.927294 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.000668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1303/2579] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:59.010608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.012276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.072952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439b749-498c-438b-ab24-2098dd92afac admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.077318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.077934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.078193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.078480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.089021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.090465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.100742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1277/2580] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:59.398557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.400129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.414343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03947e-9bec-4a79-bf83-9254f8c9ddb8 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.420436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.421167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.421501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.421860 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] PATH_INFO: `/v3/projects/2515b35c728446fdbf64358f24b1d1f3` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.423206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2515b35c728446fdbf64358f24b1d1f3)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.425691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.476647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2515b35c728446fdbf64358f24b1d1f3', 'request_id': 'req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.482786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2515b35c728446fdbf64358f24b1d1f3', 'request_id': 'req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.488530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2515b35c728446fdbf64358f24b1d1f3', 'request_id': 'req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.503136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] The token cache is being invalidate because project 2515b35c728446fdbf64358f24b1d1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:02:59.503427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.504162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.505795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1304/2581] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:59 2022] DELETE /identity/v3/projects/2515b35c728446fdbf64358f24b1d1f3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:59.510044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.520440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa233-ccdd-48cb-8182-d97b2180e96c service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.581866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.582508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.582787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.583089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] PATH_INFO: `/v3/users/d75475583a2c451a85dd90de1615f4c4` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.584306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75475583a2c451a85dd90de1615f4c4)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.586243 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.633493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75475583a2c451a85dd90de1615f4c4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.640449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75475583a2c451a85dd90de1615f4c4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.649684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75475583a2c451a85dd90de1615f4c4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.650287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75475583a2c451a85dd90de1615f4c4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.662092 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.663722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75475583a2c451a85dd90de1615f4c4'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:02:59.672536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1278/2582] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:02:59 2022] DELETE /identity/v3/users/d75475583a2c451a85dd90de1615f4c4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:59.999714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.001308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:00.015715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2dba-b9e6-4b3a-afb5-752fd6270ffb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.050934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.080770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.081878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.082156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.082429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] PATH_INFO: `/v3/projects/f43e77776607462fb2dfd746d7bca3c9` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.083734 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43e77776607462fb2dfd746d7bca3c9)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.085674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.135171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43e77776607462fb2dfd746d7bca3c9', 'request_id': 'req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.141985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43e77776607462fb2dfd746d7bca3c9', 'request_id': 'req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.148778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43e77776607462fb2dfd746d7bca3c9', 'request_id': 'req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.164461 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] The token cache is being invalidate because project f43e77776607462fb2dfd746d7bca3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:00.164828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.165708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.168092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1305/2583] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:02:59 2022] DELETE /identity/v3/projects/f43e77776607462fb2dfd746d7bca3c9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:00.172356 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.173773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:00.188317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85fb5-f65a-4eae-8f0a-b25bfd3f44e6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.252144 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.253049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.253349 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.253624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] PATH_INFO: `/v3/users/d5310603cbde423e9c2bb838cdd24915` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.254855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5310603cbde423e9c2bb838cdd24915)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.256869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.305063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5310603cbde423e9c2bb838cdd24915'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.312378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5310603cbde423e9c2bb838cdd24915'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.322470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5310603cbde423e9c2bb838cdd24915'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.323136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5310603cbde423e9c2bb838cdd24915'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.338617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-040b507e-60da-4d6a-8bb6-a6d9f240e857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5310603cbde423e9c2bb838cdd24915'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.347632 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1279/2584] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:00 2022] DELETE /identity/v3/users/d5310603cbde423e9c2bb838cdd24915 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:00.629195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.630659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:00.643628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8090b-b908-4a17-9ff8-716b7e9dd1fd admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.704649 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.706216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.706503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.706751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] PATH_INFO: `/v3/projects/67ad5f47152b4544bc863f8b75b43825` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.707996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ad5f47152b4544bc863f8b75b43825)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.709921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.760799 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ad5f47152b4544bc863f8b75b43825', 'request_id': 'req-bf239636-6f4a-45b6-b49d-fc62e85c6907'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.767454 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ad5f47152b4544bc863f8b75b43825', 'request_id': 'req-bf239636-6f4a-45b6-b49d-fc62e85c6907'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.774999 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ad5f47152b4544bc863f8b75b43825', 'request_id': 'req-bf239636-6f4a-45b6-b49d-fc62e85c6907'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.785195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.790995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] The token cache is being invalidate because project 67ad5f47152b4544bc863f8b75b43825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:00.791309 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.792159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-bf239636-6f4a-45b6-b49d-fc62e85c6907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:00.794574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1306/2585] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:00 2022] DELETE /identity/v3/projects/67ad5f47152b4544bc863f8b75b43825 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:03:00.801571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.801876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.802193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.827514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:00.828994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:00.895422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1280/2586] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:00.903557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.903751 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.904060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.926201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:00.927516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:00.938222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1307/2587] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:00.943081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.954359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d7f1-1c9a-47e4-82b2-48a834726a8d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.960819 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.961428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.961719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.961970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.962622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:00.962929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.964599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.978488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1281/2588] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.983848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.996619 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe87a80-00f1-45fb-9692-015b342ae56b None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.004204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.004809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.005068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.005333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.005969 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.007936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.046565 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1308/2589] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:00 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:01.051122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.056482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:01.065212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b131dd7-28e6-465d-90e2-31a4263f0a71 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.127657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.128364 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.128640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.128909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.129540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.131479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.157639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1282/2590] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:01.163811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.177995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe46c-e09f-493c-921b-e6c01d2b9ae7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.183644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.184255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.184529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.184795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.185441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.187198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.198646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1309/2591] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:01.202961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.214227 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693fe6-d5b5-4ba7-8b35-6e24cd385deb admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.222246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.222860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.223167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.223439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.224084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.225721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329506d3-7eb3-4aa7-a73e-32a65f4a3033 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.236835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1283/2592] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:01.241314 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.252187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6140-478a-45d8-aef5-06358f32e057 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.258159 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.258865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.259165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.259436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] PATH_INFO: `/v3/projects/c348bdc2421e426c8237ed8a5f4bc44a/users/5d62b191ac4c44728d2111de47a38cb8/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.275297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c348bdc2421e426c8237ed8a5f4bc44a, user_id=5d62b191ac4c44728d2111de47a38cb8, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.277946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.299882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1310/2593] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:01 2022] PUT /identity/v3/projects/c348bdc2421e426c8237ed8a5f4bc44a/users/5d62b191ac4c44728d2111de47a38cb8/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:01.311720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.312021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.312289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.341301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] MFA Rules not processed for user `5d62b191ac4c44728d2111de47a38cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:01.342503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.407999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1284/2594] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:01.420447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.422348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:01.481728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1fcc7-7d85-4906-9468-4375698d37f8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.485713 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97d15d2823be4939a617603cfdad0aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.486323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.486594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.486943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.499782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.502093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.518076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1311/2595] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:11.405090 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.509973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.519750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.521488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:16.589908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f52de-ee50-450f-a9a1-ca21a377c0bb None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.595833 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.596773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.597198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.597530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.600917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:16.668416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.669746 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.679998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1285/2596] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:19.821049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.823172 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:19.856786 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b78f89-7a5b-469b-934a-0756089d2977 service glance] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.865823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.866422 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.866864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.867622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] PATH_INFO: `/v3/users/f2a8b56e48b445709054849f4893767a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.868851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a8b56e48b445709054849f4893767a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.871868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.934073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a8b56e48b445709054849f4893767a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:19.941189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a8b56e48b445709054849f4893767a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:19.941864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a8b56e48b445709054849f4893767a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:19.953682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a8b56e48b445709054849f4893767a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:19.962458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-9c4ffe53-872f-4edd-87e2-a1463ac40e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a8b56e48b445709054849f4893767a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:19.968715 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1312/2597] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:19 2022] DELETE /identity/v3/users/f2a8b56e48b445709054849f4893767a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.155309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.166029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24f8f2-9156-45a8-8d3f-19de940bce29 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.231203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.231838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.232117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.232394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] PATH_INFO: `/v3/projects/e36d98f444ec4d12a1a55ef450de63a5` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.233507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36d98f444ec4d12a1a55ef450de63a5)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.235681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.288643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36d98f444ec4d12a1a55ef450de63a5', 'request_id': 'req-40fd354b-dfe2-4687-9d32-16b6e2c95a89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:20.290646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.296273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36d98f444ec4d12a1a55ef450de63a5', 'request_id': 'req-40fd354b-dfe2-4687-9d32-16b6e2c95a89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:20.305752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36d98f444ec4d12a1a55ef450de63a5', 'request_id': 'req-40fd354b-dfe2-4687-9d32-16b6e2c95a89'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:20.317657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] The token cache is being invalidate because project e36d98f444ec4d12a1a55ef450de63a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:20.317946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:20.318576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-40fd354b-dfe2-4687-9d32-16b6e2c95a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:20.320082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1286/2598] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:20 2022] DELETE /identity/v3/projects/e36d98f444ec4d12a1a55ef450de63a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.325353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.325636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.325936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.348010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:20.349215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:20.436620 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1313/2599] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:20.446340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.446621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.446903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.468578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:20.470052 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:20.479187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1287/2600] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:20.483731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.494111 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04eb84-9658-45f3-b5cd-a7cbe8b3df60 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.499106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.499750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.500105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.500382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.501297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.501594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.503328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.515636 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1314/2601] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.520423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.530313 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b750479-997a-4012-975f-cb2f962f0238 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.536634 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.537257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.537542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.537808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.538616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.540639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.571592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1288/2602] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:20.575882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.586728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d08d-b178-4324-899e-517ab19a6f8a admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.646045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.646674 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.646945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.647247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.647984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.649926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.658268 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.678942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1315/2603] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:20.680446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.692996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26675-fbc3-4b2e-b75e-e9e89ded965a admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.698308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.698923 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.699235 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.699507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.700196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.702039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.713195 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1289/2604] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:20.717471 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.728030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d7537-158c-4585-a0e0-4b4c18354f13 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.732752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.733451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.733795 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.734163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.735194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.736988 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69074-173d-4c02-9b21-b20698a56b71 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.750746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1316/2605] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:20.760149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.773939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804bd1d-3ae9-4441-a250-78b6858c63a6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.780331 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.781152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.781700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.782042 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] PATH_INFO: `/v3/projects/24d673bbd166418785b7375f67683e75/users/ae9e0b6bc4fd4dd69a8776a82cfa2273/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.799504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d673bbd166418785b7375f67683e75, user_id=ae9e0b6bc4fd4dd69a8776a82cfa2273, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.802332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.819616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1290/2606] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:20 2022] PUT /identity/v3/projects/24d673bbd166418785b7375f67683e75/users/ae9e0b6bc4fd4dd69a8776a82cfa2273/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.830380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.830909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.830909 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.860545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] MFA Rules not processed for user `ae9e0b6bc4fd4dd69a8776a82cfa2273`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:20.861767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:20.928889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1317/2607] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:20.941639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.944006 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:21.004341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9165-ce52-4491-88ba-c1ff22fd4d03 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.008380 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.008987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.009268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.009541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.018357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.019834 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b95e9-0448-45e9-bb54-5765eac9d91b service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.028700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1291/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:28.616939 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.618641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.689719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416614a-0e25-4509-a298-69956c39dae0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.695517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.696167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.696449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.696724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] PATH_INFO: `/v3/users/5d62b191ac4c44728d2111de47a38cb8` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.698040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d62b191ac4c44728d2111de47a38cb8)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.700592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.764128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d62b191ac4c44728d2111de47a38cb8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:28.771306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d62b191ac4c44728d2111de47a38cb8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:28.772042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d62b191ac4c44728d2111de47a38cb8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:28.782525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.784837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d62b191ac4c44728d2111de47a38cb8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:28.793682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d62b191ac4c44728d2111de47a38cb8'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:28.800286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1318/2609] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:28 2022] DELETE /identity/v3/users/5d62b191ac4c44728d2111de47a38cb8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:28.804970 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.805479 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.805588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.825444 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:28.826647 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.891694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1292/2610] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:28.901760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.903234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.965009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae26f5-4bf3-42ab-ae94-c68af840c408 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.969495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.970105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.970380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.970647 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.981671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.984192 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.997493 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1319/2611] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:29.284271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.285761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:29.299760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd7258-fb13-4ae4-a7fd-5a372965546c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.305065 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.305835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.306211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.306879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] PATH_INFO: `/v3/projects/c348bdc2421e426c8237ed8a5f4bc44a` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.308081 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c348bdc2421e426c8237ed8a5f4bc44a)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.309995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.326268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.358764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c348bdc2421e426c8237ed8a5f4bc44a', 'request_id': 'req-029135a6-dc46-45b1-9416-2613dedca5be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.364796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c348bdc2421e426c8237ed8a5f4bc44a', 'request_id': 'req-029135a6-dc46-45b1-9416-2613dedca5be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.373711 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c348bdc2421e426c8237ed8a5f4bc44a', 'request_id': 'req-029135a6-dc46-45b1-9416-2613dedca5be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.385528 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] The token cache is being invalidate because project c348bdc2421e426c8237ed8a5f4bc44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:29.385818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.386603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-029135a6-dc46-45b1-9416-2613dedca5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:29.388241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1293/2612] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:29 2022] DELETE /identity/v3/projects/c348bdc2421e426c8237ed8a5f4bc44a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:31.767396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.769051 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:31.782742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158341e-5e0a-42da-a11f-cc49c485e904 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.844908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.845701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.846044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.846406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] PATH_INFO: `/v3/users/ae9e0b6bc4fd4dd69a8776a82cfa2273` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.848203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9e0b6bc4fd4dd69a8776a82cfa2273)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.851242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.899209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9e0b6bc4fd4dd69a8776a82cfa2273'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:31.907094 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9e0b6bc4fd4dd69a8776a82cfa2273'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:31.907770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9e0b6bc4fd4dd69a8776a82cfa2273'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:31.921395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9e0b6bc4fd4dd69a8776a82cfa2273'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:31.932211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9e0b6bc4fd4dd69a8776a82cfa2273'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:31.938804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1320/2613] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:31 2022] DELETE /identity/v3/users/ae9e0b6bc4fd4dd69a8776a82cfa2273 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:31.943599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.944138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.944138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.964559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:31.965774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.029785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1294/2614] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:32.040407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.042009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.054328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.104678 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af67dc-423f-4aa2-8691-dea3a99e6dd2 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.109416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.110028 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.110297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.110560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.120666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.122105 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da7e6-2d17-4396-bd14-9c0fd191ab52 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.131612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1321/2615] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:32.413955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.415972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.433941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2417d5-1115-4961-8e0b-322e51059c6c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.439601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.440280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.440559 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.440837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] PATH_INFO: `/v3/projects/24d673bbd166418785b7375f67683e75` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.442091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d673bbd166418785b7375f67683e75)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.444872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.494608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d673bbd166418785b7375f67683e75', 'request_id': 'req-90dabc3d-7873-4458-a133-004195da7d99'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:32.500540 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d673bbd166418785b7375f67683e75', 'request_id': 'req-90dabc3d-7873-4458-a133-004195da7d99'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:32.510569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d673bbd166418785b7375f67683e75', 'request_id': 'req-90dabc3d-7873-4458-a133-004195da7d99'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:32.527147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] The token cache is being invalidate because project 24d673bbd166418785b7375f67683e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:32.527474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:32.528545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-90dabc3d-7873-4458-a133-004195da7d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:32.530719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1295/2616] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:32 2022] DELETE /identity/v3/projects/24d673bbd166418785b7375f67683e75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:32.537934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.538126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.538406 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.560548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:32.561430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.625719 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1322/2617] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:32.635193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.635451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.635726 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.656425 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:32.657637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.666470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1296/2618] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:32.670840 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.681174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b343-31d6-4078-9007-c2f89f7a24d2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.686676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.687308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.687573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.687866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.688485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:32.688813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.690416 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.704776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1323/2619] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:32.709640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.719839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225b30a-72cf-4369-94ec-9f41b7cb0308 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.727231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.728624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.728624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.728624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.729498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.731473 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.754523 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.765095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1297/2620] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:32.769161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.779535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3c5d-ba00-4ba1-88c4-a2af7e1b5cb8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.840013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.840637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.840942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.841253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.842157 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.845420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.875866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1324/2621] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:32.880151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.891782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab441508-c8b2-45eb-bd16-6932ec4de165 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.898122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.898775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.899082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.899355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.899999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.901610 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.912098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1298/2622] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:32.917535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.928273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e629e3a-0e10-45cc-bc36-1a5414cffcda admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.933970 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.934577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.934854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.935176 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.935820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.937418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30c7ba-b3b4-4482-99e2-4b40b16c9991 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.948489 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1325/2623] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:32.953193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.964127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543cb5e-4a91-4613-8cfe-7307776bbc4e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.969879 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.970531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.970805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.971103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] PATH_INFO: `/v3/projects/9d83c66da7a54a08a3f3e5554310e36d/users/7360c3275fa44ec0a849a8e41e589c8b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.987568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d83c66da7a54a08a3f3e5554310e36d, user_id=7360c3275fa44ec0a849a8e41e589c8b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.990432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.005990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1299/2624] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:32 2022] PUT /identity/v3/projects/9d83c66da7a54a08a3f3e5554310e36d/users/7360c3275fa44ec0a849a8e41e589c8b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:33.018964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.019358 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.019626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.053814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] MFA Rules not processed for user `7360c3275fa44ec0a849a8e41e589c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:33.055052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.061320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.127066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1326/2625] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:33.134296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.200995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dd612-930d-4ce9-b859-f10978678bee admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.208924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.210207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.210764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.211299 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.212342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.215522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.249093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1300/2626] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:33.253270 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.264386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa82d1-c108-46ce-bd2e-1bae46e4df11 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.323903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.324525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.324806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.325078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.325711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.327688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.357878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1327/2627] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:33.361850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.372225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a61d0-4463-4617-9801-ffb08c116f9f admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.376929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.377592 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.377811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.378082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.378754 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.380387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.391869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1301/2628] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:33.396475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.408360 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb074a-06c2-4ed6-8594-71652a49ca95 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.414186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.414864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.415183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.415458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] PATH_INFO: `/v3/projects/f63adbd25f754ec1a0025f5adc3eae04/users/fc9a615919e64ac383c227c4b3a7583e/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.431903 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63adbd25f754ec1a0025f5adc3eae04, user_id=fc9a615919e64ac383c227c4b3a7583e, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.434618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.450741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1328/2629] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:33 2022] PUT /identity/v3/projects/f63adbd25f754ec1a0025f5adc3eae04/users/fc9a615919e64ac383c227c4b3a7583e/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:03:33.455166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.465987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6386c62-8f6c-4903-a34c-ec3fd5567b75 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.525204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.525805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.526082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.526355 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.526988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.528614 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b803d-5464-413e-a1fd-1c01587233c9 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.538733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1302/2630] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:33.543180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.554833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46c6d0-9502-41ca-beab-2d1ec24d4754 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.560225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.560887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.561161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.561439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] PATH_INFO: `/v3/projects/f63adbd25f754ec1a0025f5adc3eae04/users/fc9a615919e64ac383c227c4b3a7583e/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.563429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63adbd25f754ec1a0025f5adc3eae04, user_id=fc9a615919e64ac383c227c4b3a7583e, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.566061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.576332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1329/2631] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:33 2022] PUT /identity/v3/projects/f63adbd25f754ec1a0025f5adc3eae04/users/fc9a615919e64ac383c227c4b3a7583e/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:33.588117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.588571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.589011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.627538 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] MFA Rules not processed for user `fc9a615919e64ac383c227c4b3a7583e`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:33.628837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.650131 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.712433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1303/2632] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:33.729488 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.732607 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.798591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643d2e0-85be-4fb3-82e9-d445f7f0a30c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.803256 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.803960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.804209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.804502 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.815186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.816750 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8adc7-9183-476f-8e90-768a30f3053f service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.826338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1330/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:34.259825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.322728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036ccd7-d541-4ec6-bbb9-c7aa6a6596a7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.327691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.328303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.328582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.328852 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] PATH_INFO: `/v3/users/7360c3275fa44ec0a849a8e41e589c8b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.330089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7360c3275fa44ec0a849a8e41e589c8b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.332049 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.385337 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7360c3275fa44ec0a849a8e41e589c8b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:34.394189 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7360c3275fa44ec0a849a8e41e589c8b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:34.405134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7360c3275fa44ec0a849a8e41e589c8b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:34.405849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7360c3275fa44ec0a849a8e41e589c8b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:34.418675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7360c3275fa44ec0a849a8e41e589c8b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:34.424983 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1304/2634] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:34 2022] DELETE /identity/v3/users/7360c3275fa44ec0a849a8e41e589c8b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:34.429926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.430214 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.430479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.438134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.452618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:34.453842 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:34.521208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1331/2635] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:34.532542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.533978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:34.577261 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.596682 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2b00b-538d-4a27-ada0-1c7be7623551 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.602815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.603455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.603734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.603998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.614149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.616039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.626644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1305/2636] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:35.178241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:35.179897 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:35.194206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552746f1-b477-4c5a-9f89-449c17c6e9b5 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:35.199629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:35.200187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.200455 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.200720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] PATH_INFO: `/v3/projects/9d83c66da7a54a08a3f3e5554310e36d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.201857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d83c66da7a54a08a3f3e5554310e36d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:35.203816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:35.253317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d83c66da7a54a08a3f3e5554310e36d', 'request_id': 'req-ee4e899e-a3cd-4a7c-ae77-071a074308eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.260056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d83c66da7a54a08a3f3e5554310e36d', 'request_id': 'req-ee4e899e-a3cd-4a7c-ae77-071a074308eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.266202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d83c66da7a54a08a3f3e5554310e36d', 'request_id': 'req-ee4e899e-a3cd-4a7c-ae77-071a074308eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.281849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] The token cache is being invalidate because project 9d83c66da7a54a08a3f3e5554310e36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:35.282210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.282880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.284581 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1332/2637] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:35 2022] DELETE /identity/v3/projects/9d83c66da7a54a08a3f3e5554310e36d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:35.290563 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:35.305759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66de36-c058-4a01-a51d-1082f021474d service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:35.375822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:35.376683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.376998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.377372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] PATH_INFO: `/v3/users/fc9a615919e64ac383c227c4b3a7583e` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.379055 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9a615919e64ac383c227c4b3a7583e)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:35.381851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:35.433909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9a615919e64ac383c227c4b3a7583e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.440974 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9a615919e64ac383c227c4b3a7583e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.449443 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9a615919e64ac383c227c4b3a7583e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.450041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9a615919e64ac383c227c4b3a7583e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.461899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2ef75088-94d7-4c21-b008-66195b725b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9a615919e64ac383c227c4b3a7583e'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.467859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1306/2638] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:35 2022] DELETE /identity/v3/users/fc9a615919e64ac383c227c4b3a7583e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:35.745207 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:35.746718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:35.762481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e899e-a3cd-4a7c-ae77-071a074308eb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:35.825446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:35.826026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.826299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.826569 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] PATH_INFO: `/v3/projects/f63adbd25f754ec1a0025f5adc3eae04` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.827707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63adbd25f754ec1a0025f5adc3eae04)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:35.829672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:35.871622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:35.889238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63adbd25f754ec1a0025f5adc3eae04', 'request_id': 'req-b78558a4-d2cd-4637-b232-7c55120b1afe'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.895553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63adbd25f754ec1a0025f5adc3eae04', 'request_id': 'req-b78558a4-d2cd-4637-b232-7c55120b1afe'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.901630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63adbd25f754ec1a0025f5adc3eae04', 'request_id': 'req-b78558a4-d2cd-4637-b232-7c55120b1afe'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.914833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] The token cache is being invalidate because project f63adbd25f754ec1a0025f5adc3eae04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:35.915148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.915834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-b78558a4-d2cd-4637-b232-7c55120b1afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:35.917351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1333/2639] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:35 2022] DELETE /identity/v3/projects/f63adbd25f754ec1a0025f5adc3eae04 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:35.922318 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.922607 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.922899 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.944244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:35.945462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:36.001284 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.014237 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1307/2640] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:36.022876 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.023199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.023476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.043878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:36.045129 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:36.054720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1334/2641] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:36.059603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.070340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514a3f1-a5b7-4390-9529-a861431371a8 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.076153 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.076774 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.077120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.077457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.078207 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:36.078513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.080180 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.093350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1308/2642] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:36.097810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.108249 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914ce37-4d0a-40e5-9b90-f300dfa3c9ef None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.113424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.114038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.114340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.114728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.115276 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.117187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.145244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1335/2643] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:36.149826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.160558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b95f6-11ad-4fbc-bf0b-9e67e763c2aa admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.219902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.220601 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.220952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.221225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.221992 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.223887 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.251913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1309/2644] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:36.256533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.267914 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d034f4-5be3-47a1-b38b-74a6919e307f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.272708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.273385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.273794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.274125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.274995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.276763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.289162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1336/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:36.293210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.311064 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6b5a1-944a-4f1f-b0db-78ce6864d376 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.312995 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.314086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.314652 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.314928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.315683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.317275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4448278-d384-4f1e-aa9d-315597dfb045 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.328421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1310/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:36.332759 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.344512 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1854e-bee9-4ca2-a2a8-d7723bc25128 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.349296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.349957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.350230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.350495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] PATH_INFO: `/v3/projects/b84b6b0f3bbd444ba79bdfc999ac47c9/users/680c717895e0471e8adca5fbf1115d46/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.366046 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84b6b0f3bbd444ba79bdfc999ac47c9, user_id=680c717895e0471e8adca5fbf1115d46, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.369348 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.383968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1337/2647] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:36 2022] PUT /identity/v3/projects/b84b6b0f3bbd444ba79bdfc999ac47c9/users/680c717895e0471e8adca5fbf1115d46/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:36.394593 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.394886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.395199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.425892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] MFA Rules not processed for user `680c717895e0471e8adca5fbf1115d46`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:36.427159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:36.506306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1311/2648] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:36.511152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.578861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c765294-5ce5-4bef-99c6-de94866e11d6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.584229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.584868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.585146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.585415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.586063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.588560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.618227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1338/2649] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:36.622577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.634274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c76e0e-d454-4512-956a-6972755e35e8 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.690408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.701258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.702277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.702561 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.702851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.703891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.707040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.737759 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1312/2650] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:36.742089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.753346 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ada8f-b793-4303-b3b5-d338a30bb802 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.762648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.763552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.763895 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.764151 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.765108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.767564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.781161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1339/2651] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:36.785660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.796305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c9f24-af00-4c08-978c-9ac75398e3a5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.801723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.802375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.802649 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.802925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] PATH_INFO: `/v3/projects/67b66e8d6bee4a919179d15820d8c2c1/users/b1c95e7399ec4bc2b3dc7e1768ca95dd/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.819067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b66e8d6bee4a919179d15820d8c2c1, user_id=b1c95e7399ec4bc2b3dc7e1768ca95dd, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.822217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.836675 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1313/2652] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:36 2022] PUT /identity/v3/projects/67b66e8d6bee4a919179d15820d8c2c1/users/b1c95e7399ec4bc2b3dc7e1768ca95dd/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:36.842564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.856419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b6dad-ae0d-4a49-90b0-96d64d343f35 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.881064 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.915676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.916289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.916564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.916857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.917472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.919171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44524bf9-bc07-4ea6-9a70-9cd6f947c05a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.929024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1340/2653] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:36.934007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.944417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848edf5d-8074-432d-a7c1-e10a1879638e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.949566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.950241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.950515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.950787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] PATH_INFO: `/v3/projects/67b66e8d6bee4a919179d15820d8c2c1/users/b1c95e7399ec4bc2b3dc7e1768ca95dd/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.952776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b66e8d6bee4a919179d15820d8c2c1, user_id=b1c95e7399ec4bc2b3dc7e1768ca95dd, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.955450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.966615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1314/2654] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:36 2022] PUT /identity/v3/projects/67b66e8d6bee4a919179d15820d8c2c1/users/b1c95e7399ec4bc2b3dc7e1768ca95dd/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:03:36.977994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.978275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.978535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.006535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] MFA Rules not processed for user `b1c95e7399ec4bc2b3dc7e1768ca95dd`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:37.007776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:37.087290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1341/2655] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:37.099407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.101379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:37.163273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264a2a0-7aec-4d5a-b74c-9e8420909baf admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.167471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.168079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.168348 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.168619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.178519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.180105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e7c0-febb-4920-92f4-29afe3037524 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.190591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1315/2656] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:37.675979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.746997 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890dc43-36ac-49db-ae07-e70a9ef53101 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.751654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.752407 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.752694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.752984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] PATH_INFO: `/v3/users/680c717895e0471e8adca5fbf1115d46` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.754201 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=680c717895e0471e8adca5fbf1115d46)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.756147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.813729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680c717895e0471e8adca5fbf1115d46'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:37.815833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.822917 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680c717895e0471e8adca5fbf1115d46'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:37.823763 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680c717895e0471e8adca5fbf1115d46'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:37.836431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680c717895e0471e8adca5fbf1115d46'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:37.844640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680c717895e0471e8adca5fbf1115d46'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:37.850862 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1342/2657] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:37 2022] DELETE /identity/v3/users/680c717895e0471e8adca5fbf1115d46 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:37.855574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.856114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.856114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.875179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:37.876374 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:37.934587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1316/2658] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:37.944530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.946671 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.012879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e399b-5c3c-44e2-baa2-d41fa8ba3c3b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.016680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.017296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.017577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.017858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.029568 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.030995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.041847 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1343/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:38.355138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.356569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.375323 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abe80a-bde3-4322-b0e7-1550c7b8856c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.381700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.382426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.382742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.382940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] PATH_INFO: `/v3/projects/b84b6b0f3bbd444ba79bdfc999ac47c9` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.385539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84b6b0f3bbd444ba79bdfc999ac47c9)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.387505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.437488 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.440741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84b6b0f3bbd444ba79bdfc999ac47c9', 'request_id': 'req-c64ec95a-613f-4796-bce9-a7842cb69836'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.448004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84b6b0f3bbd444ba79bdfc999ac47c9', 'request_id': 'req-c64ec95a-613f-4796-bce9-a7842cb69836'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.456760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84b6b0f3bbd444ba79bdfc999ac47c9', 'request_id': 'req-c64ec95a-613f-4796-bce9-a7842cb69836'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.468157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] The token cache is being invalidate because project b84b6b0f3bbd444ba79bdfc999ac47c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:38.468437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.469048 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.470533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1317/2660] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:38 2022] DELETE /identity/v3/projects/b84b6b0f3bbd444ba79bdfc999ac47c9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:38.474658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.484446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354900df-395d-45f0-8d8e-c4fffeb97a9b service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.549305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.549979 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.550255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.550527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] PATH_INFO: `/v3/users/b1c95e7399ec4bc2b3dc7e1768ca95dd` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.551776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c95e7399ec4bc2b3dc7e1768ca95dd)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.553834 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.604742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c95e7399ec4bc2b3dc7e1768ca95dd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.611549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c95e7399ec4bc2b3dc7e1768ca95dd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.612198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c95e7399ec4bc2b3dc7e1768ca95dd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.624093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.625560 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c95e7399ec4bc2b3dc7e1768ca95dd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.634015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-185e1f7d-ea86-4d77-a7ae-a4a2cafdc63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c95e7399ec4bc2b3dc7e1768ca95dd'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:38.641126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1344/2661] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:38 2022] DELETE /identity/v3/users/b1c95e7399ec4bc2b3dc7e1768ca95dd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:38.925467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.927103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.968686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ec95a-613f-4796-bce9-a7842cb69836 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.074127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.074860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.075166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.075433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] PATH_INFO: `/v3/projects/67b66e8d6bee4a919179d15820d8c2c1` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.076588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b66e8d6bee4a919179d15820d8c2c1)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.078531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.128743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b66e8d6bee4a919179d15820d8c2c1', 'request_id': 'req-b569cb15-56a2-4772-abd8-0141c4f8678c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.134408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b66e8d6bee4a919179d15820d8c2c1', 'request_id': 'req-b569cb15-56a2-4772-abd8-0141c4f8678c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.143295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b66e8d6bee4a919179d15820d8c2c1', 'request_id': 'req-b569cb15-56a2-4772-abd8-0141c4f8678c'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.160577 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] The token cache is being invalidate because project 67b66e8d6bee4a919179d15820d8c2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:39.160951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.162122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b569cb15-56a2-4772-abd8-0141c4f8678c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:39.164259 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1318/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:38 2022] DELETE /identity/v3/projects/67b66e8d6bee4a919179d15820d8c2c1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:39.169601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.170139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.170139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.229724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:39.230964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.295180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1345/2663] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:39.303155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.303440 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.303718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.312585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.327465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:39.328688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.336964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1319/2664] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:39.341369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.351394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa28db-5bf9-4f52-b444-549ed2cf5c9d None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.356660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.357261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.357535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.357796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.358448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:39.358741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.360373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.372951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1346/2665] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:39.377530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.387544 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acaec2-a0b6-4d0f-9bda-392fc5241a33 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.393624 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.394233 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.394502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.394767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.395432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.397329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.431151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1320/2666] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:39.437259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.447691 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ab40-2814-4d0d-9640-866d78c5d54e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.509507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.510131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.510386 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.510700 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.511328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.513354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.538039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1347/2667] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:39.543192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.557897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552e58-fd9a-4d85-9262-29ecf0665c28 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.563154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.563760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.564038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.564306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.564971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.566567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.577086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1321/2668] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:39.581733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.593152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe487ee-6dd1-4ed5-9a1e-842e9091c962 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.598374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.599024 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.599321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.599583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.600230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.601790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8a9f-5a64-4a2f-8c18-b3f4d2230f28 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.612688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1348/2669] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:39.617506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.629343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019d4e-6a4d-44f3-bc03-92d226b80fc6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.636505 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.637844 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.638216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.638618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] PATH_INFO: `/v3/projects/f7513d622d1d4580a54b18972639f8f6/users/18eb9e0ed8744c39bc464be346fa3e08/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.658220 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7513d622d1d4580a54b18972639f8f6, user_id=18eb9e0ed8744c39bc464be346fa3e08, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.660920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.678103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1322/2670] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:39 2022] PUT /identity/v3/projects/f7513d622d1d4580a54b18972639f8f6/users/18eb9e0ed8744c39bc464be346fa3e08/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:39.690026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.690229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.690500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.720029 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] MFA Rules not processed for user `18eb9e0ed8744c39bc464be346fa3e08`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:39.721278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:39.734641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.791431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1349/2671] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:39.797098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.869541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d54cf-6352-4cdf-8ead-2ed062b461ed admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.874258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.874868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.875177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.875450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.876093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.877991 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.906838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1323/2672] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:39.912592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.923703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16d0fb-f356-486d-8e2c-3303648855d3 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.981423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.981938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.982216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.982491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.983175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.985091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.009588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1350/2673] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:39 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:40.013794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.024501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081737b-ef67-41dd-afe1-1eda65eea5b6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.029216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.029820 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.030099 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.030371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.031037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.032644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.043715 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1324/2674] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:40.047711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.058952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7eaee-bc0a-4774-a3be-3d7e0c50c691 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.064601 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.065228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.065507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.065778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.066423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.068069 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccd4f4-5d57-44d1-89ff-2fbfd3c5a54c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.078689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1351/2675] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:40.083039 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.093352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee43d9-a80e-47f7-977e-4348ad095cae admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.098720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.099482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.099760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.100029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] PATH_INFO: `/v3/projects/871d85e34b784f86bfa344a1b41a1a4a/users/04bb44f40b5a42d48d092b78be788150/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.114907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871d85e34b784f86bfa344a1b41a1a4a, user_id=04bb44f40b5a42d48d092b78be788150, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.117704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.136876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1325/2676] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:40 2022] PUT /identity/v3/projects/871d85e34b784f86bfa344a1b41a1a4a/users/04bb44f40b5a42d48d092b78be788150/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:40.148244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.148533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.148785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.181184 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] MFA Rules not processed for user `04bb44f40b5a42d48d092b78be788150`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:40.182774 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.252491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1352/2677] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:40.257296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.306224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.323353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9c25-c278-4da3-bf41-135dd85476d7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.329218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.329824 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.330096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.330368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.330993 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.332927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.368567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1326/2678] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:40.372738 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.384124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178577-0f84-4929-a9c9-8af0c27afa92 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.404359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.446778 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.447427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.447703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.447976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.448623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.450525 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.472648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1353/2679] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:40.476933 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.488966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff8beb-a459-4800-aa4e-a166ecf6bf19 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.494967 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.495620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.495895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.496143 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.496770 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.498421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.513751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1327/2680] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:40.518777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.534530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f45d4-ef76-4087-9734-45c11be9ec7b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.541354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.542304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.542718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.543235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] PATH_INFO: `/v3/projects/dac66e292ec34817b48ea8d78ff34389/users/89fe8db2cdcc44f095761c49a876a261/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.561387 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac66e292ec34817b48ea8d78ff34389, user_id=89fe8db2cdcc44f095761c49a876a261, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.564127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.581289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1354/2681] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:40 2022] PUT /identity/v3/projects/dac66e292ec34817b48ea8d78ff34389/users/89fe8db2cdcc44f095761c49a876a261/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:40.585555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.596547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113813fb-73fe-47e0-80e5-01b616aaa5d5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.657167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.657782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.658062 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.658336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.658978 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.660622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76ce9d-dd56-489f-9dd6-adbf69d94934 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.671496 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1328/2682] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:40.675954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.687370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962ee93-4aae-4ad9-8ec7-91b47a495dbc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.692558 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.693205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.693487 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.693756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] PATH_INFO: `/v3/projects/dac66e292ec34817b48ea8d78ff34389/users/89fe8db2cdcc44f095761c49a876a261/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.696040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac66e292ec34817b48ea8d78ff34389, user_id=89fe8db2cdcc44f095761c49a876a261, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.699135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.709561 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1355/2683] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:40 2022] PUT /identity/v3/projects/dac66e292ec34817b48ea8d78ff34389/users/89fe8db2cdcc44f095761c49a876a261/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:40.720594 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.720863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.721140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.752778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] MFA Rules not processed for user `89fe8db2cdcc44f095761c49a876a261`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:40.753965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.827232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1329/2684] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:40.839026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.841928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.922585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02519-51ff-4b1f-a9a3-47ae4bf8851d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.927442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.928063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.928343 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.928616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.938849 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.940330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e43c2-116e-4bd4-ad62-b2f9f4e0ca27 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.950222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1356/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:41.478157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.479710 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:41.491938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32db2e6-b188-4db1-9b42-dcae03f337b0 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.495866 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.496510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.496818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.497067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.540693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.557513 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.558987 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.568252 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1330/2686] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:45.548286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.548808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.548808 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.571114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] MFA Rules not processed for user `9d7babb2d721414496d45840f9253852`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:45.572536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:45.633962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1357/2687] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:45.672998 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.682629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38999b1-12f5-4e98-957a-610855ca01e6 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.687031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.687663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.687938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.688202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.748845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.750321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390ba87-b1ac-4ff8-9e67-2d18c0041acb service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.759707 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1331/2688] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:47.954431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.964305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.023255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145c4a-88f3-4b33-9fa7-c696ce784672 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.029216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.029865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.030140 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.030408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] PATH_INFO: `/v3/users/18eb9e0ed8744c39bc464be346fa3e08` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.031584 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18eb9e0ed8744c39bc464be346fa3e08)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.033496 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.082402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18eb9e0ed8744c39bc464be346fa3e08'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.101210 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18eb9e0ed8744c39bc464be346fa3e08'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.101954 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18eb9e0ed8744c39bc464be346fa3e08'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.116472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18eb9e0ed8744c39bc464be346fa3e08'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.124926 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18eb9e0ed8744c39bc464be346fa3e08'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.131550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1358/2689] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:47 2022] DELETE /identity/v3/users/18eb9e0ed8744c39bc464be346fa3e08 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:48.137215 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.137786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.137786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.157694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:48.158939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:48.224943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1332/2690] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:48.236588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.238262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:48.310238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce51f-c826-40f4-ac95-ebdfe4a1b303 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.314317 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.314957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.315308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.315579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.326106 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.327606 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.339146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1359/2691] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:48.816656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.818132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:48.831928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8abde-5ae7-4d01-b001-73f56dbd01cf None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.839503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.840838 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.841304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.841731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] PATH_INFO: `/v3/projects/f7513d622d1d4580a54b18972639f8f6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.843769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7513d622d1d4580a54b18972639f8f6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.845935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.895379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7513d622d1d4580a54b18972639f8f6', 'request_id': 'req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.900885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7513d622d1d4580a54b18972639f8f6', 'request_id': 'req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.909151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7513d622d1d4580a54b18972639f8f6', 'request_id': 'req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.921343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] The token cache is being invalidate because project f7513d622d1d4580a54b18972639f8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:48.921549 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.922191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:48.923751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1333/2692] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:48 2022] DELETE /identity/v3/projects/f7513d622d1d4580a54b18972639f8f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:48.927898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.938968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbd9e-85d4-4882-ae0d-051132f64c8b service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.975403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.030594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.031382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.031666 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.031985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] PATH_INFO: `/v3/users/04bb44f40b5a42d48d092b78be788150` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.033139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bb44f40b5a42d48d092b78be788150)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.035195 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.100808 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.102256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:49.104474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bb44f40b5a42d48d092b78be788150'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.109098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.112513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bb44f40b5a42d48d092b78be788150'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.113347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bb44f40b5a42d48d092b78be788150'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.128035 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bb44f40b5a42d48d092b78be788150'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.137934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bb44f40b5a42d48d092b78be788150'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.144531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1360/2693] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:48 2022] DELETE /identity/v3/users/04bb44f40b5a42d48d092b78be788150 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:49.170287 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cff12-5e2e-45a2-b5a2-9e3785c0a0c6 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.228676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.229457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.229794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.230067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.231918 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:49.302462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.304258 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.315033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1334/2694] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:49.535233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.536804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:49.612328 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d5e3-dc5d-4c16-ac11-909c9a226f64 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.617299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.617944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.618217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.618507 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] PATH_INFO: `/v3/projects/871d85e34b784f86bfa344a1b41a1a4a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.619968 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871d85e34b784f86bfa344a1b41a1a4a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.621722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.681369 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871d85e34b784f86bfa344a1b41a1a4a', 'request_id': 'req-96ae4a84-71b0-4271-9538-70bfac2afcb1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.683352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.693137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871d85e34b784f86bfa344a1b41a1a4a', 'request_id': 'req-96ae4a84-71b0-4271-9538-70bfac2afcb1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.700594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871d85e34b784f86bfa344a1b41a1a4a', 'request_id': 'req-96ae4a84-71b0-4271-9538-70bfac2afcb1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.715574 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] The token cache is being invalidate because project 871d85e34b784f86bfa344a1b41a1a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:49.715865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.716500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.718078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1361/2695] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:49 2022] DELETE /identity/v3/projects/871d85e34b784f86bfa344a1b41a1a4a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:49.722098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.733704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdfe18-42bd-45dc-bf31-425242766103 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.795034 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.795732 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.796020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.796293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] PATH_INFO: `/v3/users/89fe8db2cdcc44f095761c49a876a261` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.803129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fe8db2cdcc44f095761c49a876a261)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.805417 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.829267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.864459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fe8db2cdcc44f095761c49a876a261'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.872246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fe8db2cdcc44f095761c49a876a261'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.880518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fe8db2cdcc44f095761c49a876a261'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.881234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fe8db2cdcc44f095761c49a876a261'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.895697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-985b8bed-cfdc-49af-9c52-6594b2e05280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fe8db2cdcc44f095761c49a876a261'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:49.905625 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1335/2696] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:49 2022] DELETE /identity/v3/users/89fe8db2cdcc44f095761c49a876a261 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:50.312204 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.313807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.341781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae4a84-71b0-4271-9538-70bfac2afcb1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.468657 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.469514 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.469802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.470146 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] PATH_INFO: `/v3/projects/dac66e292ec34817b48ea8d78ff34389` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.471378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac66e292ec34817b48ea8d78ff34389)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.473333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.575602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac66e292ec34817b48ea8d78ff34389', 'request_id': 'req-a9d2f1d4-086e-449e-8b3c-ad5016571227'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:50.594015 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac66e292ec34817b48ea8d78ff34389', 'request_id': 'req-a9d2f1d4-086e-449e-8b3c-ad5016571227'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:50.606752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac66e292ec34817b48ea8d78ff34389', 'request_id': 'req-a9d2f1d4-086e-449e-8b3c-ad5016571227'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:50.634630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] The token cache is being invalidate because project dac66e292ec34817b48ea8d78ff34389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:50.635206 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:50.636599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a9d2f1d4-086e-449e-8b3c-ad5016571227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:50.644438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1362/2697] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:50 2022] DELETE /identity/v3/projects/dac66e292ec34817b48ea8d78ff34389 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:50.648005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.649002 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.650419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.676009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:50.677358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.753274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1336/2698] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:50.762508 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.762814 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.763128 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.784605 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:50.785820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.793934 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1363/2699] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:50.798951 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.815121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31097-4659-4c02-8543-1c31274e05e2 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.820134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.820797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.821017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.821404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.821971 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:50.822344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.824091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.836871 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1337/2700] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:50.843077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.853006 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac590bb0-1fe9-409d-8616-49b2595cccd2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.858353 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.858975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.859379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.859722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.860377 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.862301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.895517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1364/2701] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:50 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:50.896837 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.907679 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b7154-9c7a-44f0-a96d-a80382766011 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.990060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.990691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.990994 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.991487 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.992136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.994744 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.027097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1338/2702] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:50 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:51.029622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.041017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee77fdd-8d37-4f5d-a7ed-de4f4ef20536 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.048281 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.049163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.049357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.049613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.050594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.051943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.067494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1365/2703] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:51.068412 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.079531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4340d-6e89-4a97-a209-3a08cf98bc90 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.091390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.092072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.092934 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.093203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.093872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.095790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2739d-bf5d-4e0e-9a8f-c311bbdff5e2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.107372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.108264 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1339/2704] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:51.112200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.123289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231d6cd-f908-4356-a24b-23fd988dc014 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.128265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.128930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.129208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.129503 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] PATH_INFO: `/v3/projects/39e0fde5dec9434991505b779419370e/users/1691f6d8f34143db8938c4da6c018bb8/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.144944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e0fde5dec9434991505b779419370e, user_id=1691f6d8f34143db8938c4da6c018bb8, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.147930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.163522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1366/2705] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:51 2022] PUT /identity/v3/projects/39e0fde5dec9434991505b779419370e/users/1691f6d8f34143db8938c4da6c018bb8/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:51.175305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.175738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.176075 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.205477 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] MFA Rules not processed for user `1691f6d8f34143db8938c4da6c018bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:51.206720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:51.276566 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1340/2706] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:51.281234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.287196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.355248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949040f0-c6f4-4241-8282-13a56fffa6af admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.360787 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.361423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.361698 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.361971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.362793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.364652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.395943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1367/2707] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:51.405100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.417571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574c8b6-216b-4420-8670-37b884624da9 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.476982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.477608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.477891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.478164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.479201 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.481214 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.506302 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1341/2708] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:51.510861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.521677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9f841-cdc8-4ee1-bc22-e746ec0ede14 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.527376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.528098 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.528472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.528819 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.529542 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.531197 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.543227 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1368/2709] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:51.548347 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.561784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b52a-534f-41fb-b87e-5dbf1551766d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.567270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.567895 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.568175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.568448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.569097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.570716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace141a5-6451-452b-a604-2d4458bee4a8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.582224 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1342/2710] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:51.586617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.598313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e8a8-7cd5-4411-a397-f5affb33eb1e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.603928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.604945 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.605182 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.605445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] PATH_INFO: `/v3/projects/58a054d5a0344d3ea095a72658c92616/users/509e4657352147b19bb7ed54e6e4ae1b/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.621610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a054d5a0344d3ea095a72658c92616, user_id=509e4657352147b19bb7ed54e6e4ae1b, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.624372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.640804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1369/2711] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:51 2022] PUT /identity/v3/projects/58a054d5a0344d3ea095a72658c92616/users/509e4657352147b19bb7ed54e6e4ae1b/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:51.651471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.652009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.652009 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.694054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] MFA Rules not processed for user `509e4657352147b19bb7ed54e6e4ae1b`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:51.695482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:51.763164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.771163 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1343/2712] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:51.774180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.859776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff1de-014b-4bc1-887c-e119acff453d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.864540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.865164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.865475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.865742 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.866380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.868383 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.904032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1370/2713] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:51.908244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.921301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3572dfc-cf7d-4c5d-aa2a-4f0f38269921 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.990467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.991345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.992855 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.993151 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.993912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.996535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.028342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1344/2714] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:52.032127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.046580 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b71ce9-b089-4935-bd7e-74ba930a66ba admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.052841 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.053598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.054025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.054363 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.055641 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.058453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.090770 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1371/2715] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:52.092168 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.105755 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617b63-e77e-46ba-9af6-51192f6730d8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.111450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.112074 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.114193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.114193 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] PATH_INFO: `/v3/projects/5a4a937495594844a740195b48aae885/users/9ec329b3914d44edb90b743872c8d114/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.133103 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4a937495594844a740195b48aae885, user_id=9ec329b3914d44edb90b743872c8d114, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.137138 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.160584 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1345/2716] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:52 2022] PUT /identity/v3/projects/5a4a937495594844a740195b48aae885/users/9ec329b3914d44edb90b743872c8d114/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:52.161245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.174737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226568d4-d53d-4ee6-9d8b-82e65a695956 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.210884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.249890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.250533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.250817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.251134 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.251788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.253411 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c243ba-3054-4fda-bdad-89b18fbe7fe7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.264044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1372/2717] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:52.268406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.279807 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c1825-359d-4bb4-8e88-ec46a1773af2 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.284922 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.285599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.285878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.286160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] PATH_INFO: `/v3/projects/5a4a937495594844a740195b48aae885/users/9ec329b3914d44edb90b743872c8d114/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.288156 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4a937495594844a740195b48aae885, user_id=9ec329b3914d44edb90b743872c8d114, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.292269 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.303572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1346/2718] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:52 2022] PUT /identity/v3/projects/5a4a937495594844a740195b48aae885/users/9ec329b3914d44edb90b743872c8d114/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:52.376020 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.376577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.376577 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.423806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] MFA Rules not processed for user `9ec329b3914d44edb90b743872c8d114`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:52.425446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.497820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1373/2719] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:52.509452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.512115 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.572321 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898abd1-c50a-4b7f-be4d-0d6bd878dcb5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.576167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.576794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.577068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.577340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.587416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.588982 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.599721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1347/2720] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:52.646442 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.648145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.662798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abe59-7d2c-4260-8520-1c15fafc41f7 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.667703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.668391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.668745 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.669091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.750396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.752354 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.774820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1374/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:53.375310 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.376822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.466290 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.508955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774bc4b-6603-411c-97ec-6e3a7dafcac2 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.514357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.514972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.515277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.515542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.517311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.529760 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.531516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.543860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1348/2722] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:55.709547 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.711114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:55.771961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.776470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a25db-cc70-4355-8b41-a4d118ed49c8 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.780717 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.781638 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.781915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.782183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.783908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:55.854665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.861733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.863333 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1997e-4792-4e8e-9622-d82b2cf29a3d service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.869644 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272f08-9e09-4d95-b028-0ef1e9b854a4 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.874862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.875101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1375/2723] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:55.875527 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.875878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.876218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] PATH_INFO: `/v3/users/1691f6d8f34143db8938c4da6c018bb8` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.877421 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1691f6d8f34143db8938c4da6c018bb8)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.879491 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.937104 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1691f6d8f34143db8938c4da6c018bb8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:55.945446 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1691f6d8f34143db8938c4da6c018bb8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:55.958088 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1691f6d8f34143db8938c4da6c018bb8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:55.958461 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1691f6d8f34143db8938c4da6c018bb8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:55.974046 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1691f6d8f34143db8938c4da6c018bb8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:55.985863 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1349/2724] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:55 2022] DELETE /identity/v3/users/1691f6d8f34143db8938c4da6c018bb8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:03:55.987869 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.988255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.988948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.022421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:56.024384 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:56.101729 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1376/2725] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:56.112345 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.113920 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:56.183556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f91a-df9b-4150-a6a1-b72c09e40437 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.187379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.188128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.188411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.188683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.199557 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.201011 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.212109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1350/2726] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:56.543549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.545267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:56.570047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26629c-b74b-4bef-854d-e8151bd723b2 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.577947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.578458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.578818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.579327 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] PATH_INFO: `/v3/projects/39e0fde5dec9434991505b779419370e` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.580552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e0fde5dec9434991505b779419370e)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.583212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.658345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e0fde5dec9434991505b779419370e', 'request_id': 'req-edf373f3-9cf6-4a50-81a5-fd7088a07703'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.666655 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e0fde5dec9434991505b779419370e', 'request_id': 'req-edf373f3-9cf6-4a50-81a5-fd7088a07703'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.673379 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e0fde5dec9434991505b779419370e', 'request_id': 'req-edf373f3-9cf6-4a50-81a5-fd7088a07703'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.691228 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] The token cache is being invalidate because project 39e0fde5dec9434991505b779419370e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:56.691586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.692308 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.693898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1377/2727] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:56 2022] DELETE /identity/v3/projects/39e0fde5dec9434991505b779419370e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:56.698311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.710676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e112c-9e0e-42c6-bf3c-fbae99fd428a service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.747663 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.778173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.779729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.779729 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.780073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] PATH_INFO: `/v3/users/509e4657352147b19bb7ed54e6e4ae1b` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.781204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] RBAC: Authorizing `identity:delete_user(user_id=509e4657352147b19bb7ed54e6e4ae1b)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.783176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.843236 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509e4657352147b19bb7ed54e6e4ae1b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.851790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509e4657352147b19bb7ed54e6e4ae1b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.860984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509e4657352147b19bb7ed54e6e4ae1b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.862742 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509e4657352147b19bb7ed54e6e4ae1b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.876697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509e4657352147b19bb7ed54e6e4ae1b'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:56.886176 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1351/2728] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:56 2022] DELETE /identity/v3/users/509e4657352147b19bb7ed54e6e4ae1b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:57.279593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:57.281306 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:57.302084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf373f3-9cf6-4a50-81a5-fd7088a07703 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:57.344681 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:57.416807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:57.417877 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:57.418216 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:57.418557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] PATH_INFO: `/v3/projects/58a054d5a0344d3ea095a72658c92616` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:57.419868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a054d5a0344d3ea095a72658c92616)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:57.422179 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:57.495891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a054d5a0344d3ea095a72658c92616', 'request_id': 'req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.504042 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a054d5a0344d3ea095a72658c92616', 'request_id': 'req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.514260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a054d5a0344d3ea095a72658c92616', 'request_id': 'req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.531480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] The token cache is being invalidate because project 58a054d5a0344d3ea095a72658c92616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:57.531780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.532443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.534296 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1378/2729] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:57 2022] DELETE /identity/v3/projects/58a054d5a0344d3ea095a72658c92616 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:57.538320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:57.539853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:57.555077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d80d75-b3ba-416f-92b7-45a21679258c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:57.618453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:57.619134 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:57.619509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:57.619966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] PATH_INFO: `/v3/users/9ec329b3914d44edb90b743872c8d114` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:57.621365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec329b3914d44edb90b743872c8d114)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:57.623423 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:57.679507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec329b3914d44edb90b743872c8d114'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.687626 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec329b3914d44edb90b743872c8d114'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.690148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:57.696849 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec329b3914d44edb90b743872c8d114'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.697919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec329b3914d44edb90b743872c8d114'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.714230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-b873540b-8474-48e8-9e17-2944b17d1333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec329b3914d44edb90b743872c8d114'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:57.720620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1352/2730] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:57 2022] DELETE /identity/v3/users/9ec329b3914d44edb90b743872c8d114 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:58.004854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.007171 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.027874 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0cc9-3b9a-462e-8f22-e2ff2b4e0844 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.093297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.094050 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.094355 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.094597 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] PATH_INFO: `/v3/projects/5a4a937495594844a740195b48aae885` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.098677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4a937495594844a740195b48aae885)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.100724 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.161594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4a937495594844a740195b48aae885', 'request_id': 'req-ad3e0b97-476e-473d-8686-50ff35fec7c9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:58.168904 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4a937495594844a740195b48aae885', 'request_id': 'req-ad3e0b97-476e-473d-8686-50ff35fec7c9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:58.170321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:58.177961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4a937495594844a740195b48aae885', 'request_id': 'req-ad3e0b97-476e-473d-8686-50ff35fec7c9'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:58.194213 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] The token cache is being invalidate because project 5a4a937495594844a740195b48aae885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:58.194579 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:58.195299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-ad3e0b97-476e-473d-8686-50ff35fec7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:58.197090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1379/2731] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:58 2022] DELETE /identity/v3/projects/5a4a937495594844a740195b48aae885 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:58.204238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.204810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.204913 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.233960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:58.235270 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.300706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1353/2732] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:58.309825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.310305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.310305 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.331714 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:58.332958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.347790 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1380/2733] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:58.352213 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.364172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d06a60-4fe9-4457-bd51-606b969000f1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.369864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.370700 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.371454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.371454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.372124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:58.372535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.374301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.388714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1354/2734] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:58.395373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.405900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d74242-897a-44d7-a9b9-581a2c05b584 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.411055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.412026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.412364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.412703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.413430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.415996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.463376 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1381/2735] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:58.467629 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.478196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25a38e-eb46-43ea-8047-ed5a5e0b4bf8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.540891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.541704 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.542050 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.542332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.542984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.545019 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.573678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1355/2736] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:58.575205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.586103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8c6e4-0c58-432f-9bd5-68041b1222fc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.592367 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.593082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.593448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.593785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.594516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.596165 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.607090 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1382/2737] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:58.611358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.622238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd6ae8-1d16-47d5-8794-36037b9b2b42 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.627957 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.628590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.628860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.629130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.629775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.631438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9047f7-d4ba-477f-94de-5d03a9cb04e3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.643464 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1356/2738] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:03:58.647960 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.659467 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d0aa9-e263-4408-ad5b-c05100e3cb7c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.664313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.664977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.665255 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.665531 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] PATH_INFO: `/v3/projects/fdd977ae5e3d45c7b0d812d56ca08efb/users/eb87323d730a42538a94779adb249df6/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.682908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd977ae5e3d45c7b0d812d56ca08efb, user_id=eb87323d730a42538a94779adb249df6, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.686056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.701272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1383/2739] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:03:58 2022] PUT /identity/v3/projects/fdd977ae5e3d45c7b0d812d56ca08efb/users/eb87323d730a42538a94779adb249df6/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:58.712717 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.713030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.713304 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.742616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] MFA Rules not processed for user `eb87323d730a42538a94779adb249df6`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:58.746244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.784936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:58.830751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1357/2740] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:58.844408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.846235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.911815 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25fde-71c5-423b-aa1a-e851fcbf2cfc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.915419 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.916034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.916341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.916608 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.926311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.928065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752da381-e33a-4de9-a6f8-44860ce52389 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.938659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1384/2741] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:58.968878 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.062657 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eac8cb-e6e9-45aa-8257-7f8841b01683 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.068346 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.068651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.068935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.069210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] PATH_INFO: `/v3/users/eb87323d730a42538a94779adb249df6` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.070344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb87323d730a42538a94779adb249df6)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.072898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.135268 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb87323d730a42538a94779adb249df6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.142114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb87323d730a42538a94779adb249df6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.150639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb87323d730a42538a94779adb249df6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.151352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb87323d730a42538a94779adb249df6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.163646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb87323d730a42538a94779adb249df6'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.169800 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1358/2742] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:03:58 2022] DELETE /identity/v3/users/eb87323d730a42538a94779adb249df6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:59.174434 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.174720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.175026 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.205102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:59.206347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.291707 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1385/2743] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:59.303552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.305334 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.372535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.401072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d7e1-b723-4c8d-940b-2b8716f47e04 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.406459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.407401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.407678 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.407945 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.424472 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.425908 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6407397-a4f9-47bc-8657-d50016d81aba service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.441184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1359/2744] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:59.795545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.797131 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.817398 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b07b5-269a-4f30-8a88-208e26025fa0 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.826908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.828415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.828886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.829187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] PATH_INFO: `/v3/projects/fdd977ae5e3d45c7b0d812d56ca08efb` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.830788 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd977ae5e3d45c7b0d812d56ca08efb)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.833009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.851080 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.897905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd977ae5e3d45c7b0d812d56ca08efb', 'request_id': 'req-e246c4eb-7c1a-4c80-a587-12ac70767b6a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.905286 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd977ae5e3d45c7b0d812d56ca08efb', 'request_id': 'req-e246c4eb-7c1a-4c80-a587-12ac70767b6a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.911982 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd977ae5e3d45c7b0d812d56ca08efb', 'request_id': 'req-e246c4eb-7c1a-4c80-a587-12ac70767b6a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.926900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] The token cache is being invalidate because project fdd977ae5e3d45c7b0d812d56ca08efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:03:59.927258 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.927873 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e246c4eb-7c1a-4c80-a587-12ac70767b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:03:59.929453 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1386/2745] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:03:59 2022] DELETE /identity/v3/projects/fdd977ae5e3d45c7b0d812d56ca08efb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:59.934954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.935271 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.935548 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.957512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:59.958758 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:00.025102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1360/2746] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:00.034466 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.034822 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.035209 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.056991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:00.058990 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:00.068186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1387/2747] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:00.073105 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.084150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c25f6-e803-4ff6-bd7d-49d846a9fd27 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.089731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.090343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.090616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.090883 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.091558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:00.091856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.093462 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.109373 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1361/2748] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:00.110823 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.120983 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2b702-f8ec-4291-9044-36ac7a35d0bd None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.126208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.126830 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.127107 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.127429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.128052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.129957 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.160156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1388/2749] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:04:00 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:00.164436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.174587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3875fe-cf5c-410b-906c-e57d41fb66f4 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.244218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.245116 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.245320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.245600 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.246294 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.248273 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.273070 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1362/2750] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:00 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:00.278510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.290494 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345bd72-757e-44ba-be26-a16ce2496a5e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.296555 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.297198 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.297472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.297741 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.298457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.300177 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.313040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1389/2751] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:04:00.319645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.332094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c072f2-71fe-4019-a10b-281eff0462b5 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.340361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.341796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.342080 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.342542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.343242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.344882 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14869d1b-5f6d-4c7c-a501-72309864e2bd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.359418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1363/2752] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:04:00.363544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.374848 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb7919-6257-45d6-a3d6-95aa9b9cd1e4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.380614 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.381412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.381709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.381980 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] PATH_INFO: `/v3/projects/b679cbd3889c4b34af92332f60fa4bb2/users/250a64ff5e1146cb8b64b70c7887c3c5/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.404731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b679cbd3889c4b34af92332f60fa4bb2, user_id=250a64ff5e1146cb8b64b70c7887c3c5, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.407545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.423313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1390/2753] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:04:00 2022] PUT /identity/v3/projects/b679cbd3889c4b34af92332f60fa4bb2/users/250a64ff5e1146cb8b64b70c7887c3c5/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:00.434393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.434660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.434942 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.466029 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] MFA Rules not processed for user `250a64ff5e1146cb8b64b70c7887c3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:00.467322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:00.499078 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.538130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1364/2754] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:04:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:04:00.549351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.551141 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:00.612552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a03b-5203-4bdd-957c-800f68df8a1b admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.616526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.617149 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.617430 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.617701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.630273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.632971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.645265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1391/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:02.906973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.909072 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:02.973152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.974660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:03.043160 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22bd1-0706-4045-8bec-ae663dac2a5d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.052095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.052767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.053037 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.053319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.056560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:03.074521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.075874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.077579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.094510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1365/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:03.108395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa7168-cda0-4935-ba04-306d4319fcf4 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.113798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.115199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.115199 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.115418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] PATH_INFO: `/v3/users/1558d42f4c7f437495af05d4c8c42e6e` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.116477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1558d42f4c7f437495af05d4c8c42e6e)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.118456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.172236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1558d42f4c7f437495af05d4c8c42e6e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.183273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1558d42f4c7f437495af05d4c8c42e6e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.186000 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1558d42f4c7f437495af05d4c8c42e6e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.199660 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1558d42f4c7f437495af05d4c8c42e6e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.209133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1558d42f4c7f437495af05d4c8c42e6e'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.216405 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1392/2757] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:04:02 2022] DELETE /identity/v3/users/1558d42f4c7f437495af05d4c8c42e6e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:04:03.437485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.449424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46333990-2138-49b5-868f-c4bd06211f44 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.524569 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.525565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.526697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.526697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] PATH_INFO: `/v3/projects/2b0736ff08ae4fa6be73ec678c98077f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.527665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b0736ff08ae4fa6be73ec678c98077f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.529573 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.591098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b0736ff08ae4fa6be73ec678c98077f', 'request_id': 'req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.599157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b0736ff08ae4fa6be73ec678c98077f', 'request_id': 'req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.602705 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.609598 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b0736ff08ae4fa6be73ec678c98077f', 'request_id': 'req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.626752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] The token cache is being invalidate because project 2b0736ff08ae4fa6be73ec678c98077f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:04:03.627068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.627731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.633281 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1366/2758] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:04:03 2022] DELETE /identity/v3/projects/2b0736ff08ae4fa6be73ec678c98077f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:03.633572 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.635245 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:03.650273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429e69b-58e8-4ede-b070-3c5e82142a27 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.726038 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.726711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.727030 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.727299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] PATH_INFO: `/v3/users/62ec214672f54d089fce6dd5f7af19f1` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.728444 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ec214672f54d089fce6dd5f7af19f1)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.730728 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.795760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ec214672f54d089fce6dd5f7af19f1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.805269 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ec214672f54d089fce6dd5f7af19f1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.806008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ec214672f54d089fce6dd5f7af19f1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.814760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.821516 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ec214672f54d089fce6dd5f7af19f1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.831598 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ec214672f54d089fce6dd5f7af19f1'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:03.838267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1393/2759] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:04:03 2022] DELETE /identity/v3/users/62ec214672f54d089fce6dd5f7af19f1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:04.052988 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:04.054485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:04.074773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4ac34-b261-40a4-b0fa-71f7d1b17ad8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:04.150101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:04.150892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.151133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.151434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] PATH_INFO: `/v3/projects/8dd1826a770746cd93907aa70e4c3aa9` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.152690 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd1826a770746cd93907aa70e4c3aa9)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:04.154623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:04.205903 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd1826a770746cd93907aa70e4c3aa9', 'request_id': 'req-fc7d35d8-aef9-4987-80e9-2017c936a7cc'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:04.211720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd1826a770746cd93907aa70e4c3aa9', 'request_id': 'req-fc7d35d8-aef9-4987-80e9-2017c936a7cc'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:04.220671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd1826a770746cd93907aa70e4c3aa9', 'request_id': 'req-fc7d35d8-aef9-4987-80e9-2017c936a7cc'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:04.232890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] The token cache is being invalidate because project 8dd1826a770746cd93907aa70e4c3aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:04:04.233175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:04.233877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-fc7d35d8-aef9-4987-80e9-2017c936a7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:04.235508 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1367/2760] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:04:04 2022] DELETE /identity/v3/projects/8dd1826a770746cd93907aa70e4c3aa9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:10.855906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.857592 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:10.871673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed811c2-f904-4ac8-9164-bbe56db86998 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.956499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.957142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.957429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.957694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] PATH_INFO: `/v3/users/250a64ff5e1146cb8b64b70c7887c3c5` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.958857 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=250a64ff5e1146cb8b64b70c7887c3c5)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.960851 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.016060 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250a64ff5e1146cb8b64b70c7887c3c5'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.023357 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250a64ff5e1146cb8b64b70c7887c3c5'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.024085 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250a64ff5e1146cb8b64b70c7887c3c5'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.036929 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250a64ff5e1146cb8b64b70c7887c3c5'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.046262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250a64ff5e1146cb8b64b70c7887c3c5'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.052730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1394/2761] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:04:10 2022] DELETE /identity/v3/users/250a64ff5e1146cb8b64b70c7887c3c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:11.057392 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.058251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.058372 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.079305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:11.080451 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:11.151867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1368/2762] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:11.163684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.165362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:11.229002 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626f40-56ee-4be2-901f-3ee08d895e32 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.233018 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.233626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.233901 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.234168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.244818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.247332 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.261218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1395/2763] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:11.535799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.537301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:11.552238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cdcd6-b315-43b7-9d7f-72ad553aa3a4 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.557506 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.558589 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.558862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.559161 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] PATH_INFO: `/v3/projects/b679cbd3889c4b34af92332f60fa4bb2` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.560560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b679cbd3889c4b34af92332f60fa4bb2)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.562611 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.580665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.615375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b679cbd3889c4b34af92332f60fa4bb2', 'request_id': 'req-a4858040-e594-4de2-a7d9-afed73999156'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.620796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b679cbd3889c4b34af92332f60fa4bb2', 'request_id': 'req-a4858040-e594-4de2-a7d9-afed73999156'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.630394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b679cbd3889c4b34af92332f60fa4bb2', 'request_id': 'req-a4858040-e594-4de2-a7d9-afed73999156'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.642748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] The token cache is being invalidate because project b679cbd3889c4b34af92332f60fa4bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:04:11.643109 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.643817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a4858040-e594-4de2-a7d9-afed73999156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:04:11.645422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1369/2764] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:04:11 2022] DELETE /identity/v3/projects/b679cbd3889c4b34af92332f60fa4bb2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:21.255469 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:21.639378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.167351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:27.169583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:27.254436 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354c158-389c-4826-9257-da7759e78394 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.306975 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.307585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.307863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.308499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.310783 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:27.376093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:27.377562 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06390c-720c-4976-8cb8-0784209e56f4 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:27.387421 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1396/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:34.297947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:34.298567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:34.298567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:34.318469 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] MFA Rules not processed for user `455284c947254207a719459efc031bea`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:34.319880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:34.329371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1370/2766] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:04:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:37.382644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=77 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:44.313312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:44.499900 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:44.500446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:44.500446 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:44.520958 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] MFA Rules not processed for user `9d7babb2d721414496d45840f9253852`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:44.522200 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:44.579706 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1397/2767] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:04:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:53.168794 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:53.171202 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:53.248635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d698b7-d4ad-439d-b36a-925f03b0f0ac None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:53.253242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:53.253862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:53.254140 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:53.254409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:53.256722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:53.333921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:53.336066 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:53.347514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1371/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:54.571626 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=25 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:03.342363 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=50 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:43.263809 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:43.265356 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:43.279656 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4544d-3544-4806-bf23-22bf848a4e87 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:43.284162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:43.284747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:43.285022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:43.285304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:43.287212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:43.393703 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:43.395236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd14ea-2d68-4d9d-85de-1ec1cf50b44d service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:43.416479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1398/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:43.599603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:43.611383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac37a6-51b6-4832-a993-59b7ab0f0d14 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:43.615767 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:43.616379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:43.616659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:43.616935 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:43.618859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:43.633244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:43.634674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:43.646067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1372/2770] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:50.748244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:50.749266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:50.749550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:50.769765 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] MFA Rules not processed for user `9d7babb2d721414496d45840f9253852`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:50.770972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:50.779223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1399/2771] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:05:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:53.641054 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:00.764274 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=49 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:48.289169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.290643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:48.303438 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b13e9-ac17-43a2-8cf9-fcdf1c36e545 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:48.307301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:48.307915 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:48.308230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:48.308522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:48.310547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:48.376121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:48.377599 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:48.387665 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1373/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:06:55.879499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:55.881178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:55.896473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0108-af11-4e1a-9ed0-c4537364fead None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:55.901629 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:55.902392 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:55.902870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:55.903186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] PATH_INFO: `/v3/users/455284c947254207a719459efc031bea` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:55.904320 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] RBAC: Authorizing `identity:delete_user(user_id=455284c947254207a719459efc031bea)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:55.906247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:55.959194 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455284c947254207a719459efc031bea'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:55.966811 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455284c947254207a719459efc031bea'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:55.967564 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455284c947254207a719459efc031bea'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:55.984725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455284c947254207a719459efc031bea'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:55.994993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455284c947254207a719459efc031bea'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.003057 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1400/2773] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:55 2022] DELETE /identity/v3/users/455284c947254207a719459efc031bea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:56.240554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.260619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8d3bf-0771-4c4c-ac4e-95d5b6d6fa51 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.330757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.331468 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.331748 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.332024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] PATH_INFO: `/v3/projects/d4baca9d3a3e45d883482957e9ce8cb0` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.333192 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4baca9d3a3e45d883482957e9ce8cb0)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.335209 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.386199 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4baca9d3a3e45d883482957e9ce8cb0', 'request_id': 'req-9f4146d5-2575-458f-a13a-0990a0df1abd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.399783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4baca9d3a3e45d883482957e9ce8cb0', 'request_id': 'req-9f4146d5-2575-458f-a13a-0990a0df1abd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.409637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4baca9d3a3e45d883482957e9ce8cb0', 'request_id': 'req-9f4146d5-2575-458f-a13a-0990a0df1abd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.422757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] The token cache is being invalidate because project d4baca9d3a3e45d883482957e9ce8cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:56.423083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.423791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.425378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1374/2774] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:56 2022] DELETE /identity/v3/projects/d4baca9d3a3e45d883482957e9ce8cb0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:56.429526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.430916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.448370 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d377f74-8e8f-4b15-9305-dd618c5adcda admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.518041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.518687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.518961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.519265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] PATH_INFO: `/v3/users/74d3fcced3ff48f09e6f202127c503f4` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.520389 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d3fcced3ff48f09e6f202127c503f4)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.522350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.579552 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d3fcced3ff48f09e6f202127c503f4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.587600 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d3fcced3ff48f09e6f202127c503f4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.588445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d3fcced3ff48f09e6f202127c503f4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.604996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d3fcced3ff48f09e6f202127c503f4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.608113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:56.614679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d3fcced3ff48f09e6f202127c503f4'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.622290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1401/2775] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:56 2022] DELETE /identity/v3/users/74d3fcced3ff48f09e6f202127c503f4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:56.807278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:56.808892 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:56.838868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4146d5-2575-458f-a13a-0990a0df1abd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:56.863859 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:56.905930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:56.906494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:56.906775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:56.907076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] PATH_INFO: `/v3/projects/9462ffe85319404bb2bb8be3b9855efa` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:56.908352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] RBAC: Authorizing `identity:delete_project(project_id=9462ffe85319404bb2bb8be3b9855efa)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:56.910575 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:56.962424 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9462ffe85319404bb2bb8be3b9855efa', 'request_id': 'req-02e9202c-7317-49da-9206-763b16dcdade'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.969470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9462ffe85319404bb2bb8be3b9855efa', 'request_id': 'req-02e9202c-7317-49da-9206-763b16dcdade'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.984788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9462ffe85319404bb2bb8be3b9855efa', 'request_id': 'req-02e9202c-7317-49da-9206-763b16dcdade'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.998241 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] The token cache is being invalidate because project 9462ffe85319404bb2bb8be3b9855efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:56.998587 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:56.999401 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:57.001358 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1375/2776] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:56 2022] DELETE /identity/v3/projects/9462ffe85319404bb2bb8be3b9855efa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:59.388800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:59.390311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:59.409479 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cac80-5dac-4678-9734-3b8363e22884 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:59.479978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:59.480659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:59.480955 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:59.481313 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] PATH_INFO: `/v3/users/9d7babb2d721414496d45840f9253852` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:59.482501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7babb2d721414496d45840f9253852)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:59.484522 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:59.544425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7babb2d721414496d45840f9253852'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.551349 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7babb2d721414496d45840f9253852'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.552012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7babb2d721414496d45840f9253852'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.565677 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7babb2d721414496d45840f9253852'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.574831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-f8e1fa07-8642-42b4-b90b-cb237410f734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7babb2d721414496d45840f9253852'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.581388 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1402/2777] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:06:59 2022] DELETE /identity/v3/users/9d7babb2d721414496d45840f9253852 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:59.765230 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:59.766751 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:59.780128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9202c-7317-49da-9206-763b16dcdade admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:59.854659 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:59.855409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:59.855688 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:59.855962 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] PATH_INFO: `/v3/projects/fe19812dff73433b8b29d24ddbea3f56` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:59.857309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe19812dff73433b8b29d24ddbea3f56)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:59.859296 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:59.909560 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe19812dff73433b8b29d24ddbea3f56', 'request_id': 'req-cc096ca7-4957-4c06-b052-14f1a6b381be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.917060 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:59.918480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe19812dff73433b8b29d24ddbea3f56', 'request_id': 'req-cc096ca7-4957-4c06-b052-14f1a6b381be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.929776 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe19812dff73433b8b29d24ddbea3f56', 'request_id': 'req-cc096ca7-4957-4c06-b052-14f1a6b381be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.941661 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] The token cache is being invalidate because project fe19812dff73433b8b29d24ddbea3f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:06:59.941948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.942686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-cc096ca7-4957-4c06-b052-14f1a6b381be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:06:59.944219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1376/2778] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:06:59 2022] DELETE /identity/v3/projects/fe19812dff73433b8b29d24ddbea3f56 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:59.949273 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:59.949557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:59.949833 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:59.970644 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:59.971886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:00.045148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1403/2779] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:00.053535 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.053815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.054086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.076077 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:00.077218 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:00.087773 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1377/2780] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:00.093371 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.104378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990773d-bdf0-4eaa-9d7f-443743b94993 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.110646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.111277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.111553 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.111820 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.112470 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:00.112768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.114390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.121475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:00.127769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1404/2781] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:00.132543 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.142228 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8cc79-37af-4cb4-b1f7-c3cea5c35609 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.147562 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.148172 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.148450 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.148745 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.149375 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.151308 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.191615 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1378/2782] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:00.197673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.212817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282971-7277-4a5a-8876-6c8b2010fbf6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.271189 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.271784 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.272067 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.272339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.272967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.274891 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.298236 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1405/2783] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:00.303045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.315642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906799d-ac8d-428e-8beb-ba5bfa92dcdb admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.320719 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.321342 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.321653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.321927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.322570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.324206 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.335170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1379/2784] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:00.339339 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.350164 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ce8a8-88e3-402e-875e-3015f3770790 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.354965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.355602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.355880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.356153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.356803 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.358400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4480002-1ee0-411a-a710-3d18dacb6f90 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.369465 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1406/2785] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:07:00.373778 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.384024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa803-f697-4646-a539-02827dc3c409 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.388909 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.389565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.389845 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.390122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] PATH_INFO: `/v3/projects/20f88522579847c7ae4025746621a3d7/users/53f9627e13204a688cfb9fca73ac4425/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.405171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f88522579847c7ae4025746621a3d7, user_id=53f9627e13204a688cfb9fca73ac4425, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.407905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.421917 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1380/2786] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:00 2022] PUT /identity/v3/projects/20f88522579847c7ae4025746621a3d7/users/53f9627e13204a688cfb9fca73ac4425/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:07:00.433183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.433697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.433697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.464865 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] MFA Rules not processed for user `53f9627e13204a688cfb9fca73ac4425`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:00.466183 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:00.546941 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1407/2787] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:00.551617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.618671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a98f8-73a2-4eed-8f72-c1960f7c94f3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.624033 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.624640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.624921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.625196 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.625826 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.627775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.660185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1381/2788] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:00.666952 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.680321 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2c156-6ebc-48f3-ade2-b9a13097c89a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.746861 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.747603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.747879 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.748144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.748798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.750746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.761984 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:00.774971 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1408/2789] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:00.779159 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.790124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7082dc-fb58-42da-b534-43b611563df1 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.795368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.795961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.796240 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.796515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.797155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.798757 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.809885 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1382/2790] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:00.814722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.825390 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678b55-d044-4977-830b-8cdc23dbcbb1 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.830793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.831460 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.831737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.832007 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] PATH_INFO: `/v3/projects/383a13ba16a7414e9a9f54d5be7a937e/users/42f05db0e91c4f38940ec45a53366669/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.847362 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=383a13ba16a7414e9a9f54d5be7a937e, user_id=42f05db0e91c4f38940ec45a53366669, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.850045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.864616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1409/2791] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:00 2022] PUT /identity/v3/projects/383a13ba16a7414e9a9f54d5be7a937e/users/42f05db0e91c4f38940ec45a53366669/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:00.868867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.879030 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e64d6-42ab-4e21-895a-615aa69ca5f3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.928361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:00.943886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.944510 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.944791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.945068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.945714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.947402 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd60786-b61c-4dea-9bf7-28b75680845e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.957984 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1383/2792] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:07:00.962399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:00.973884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438b1d4-c5ab-432a-85f5-9d9c16b49e0f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:00.978796 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:00.979485 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.979732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.980017 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] PATH_INFO: `/v3/projects/383a13ba16a7414e9a9f54d5be7a937e/users/42f05db0e91c4f38940ec45a53366669/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.982014 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383a13ba16a7414e9a9f54d5be7a937e, user_id=42f05db0e91c4f38940ec45a53366669, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:00.984766 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:00.996040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1410/2793] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:00 2022] PUT /identity/v3/projects/383a13ba16a7414e9a9f54d5be7a937e/users/42f05db0e91c4f38940ec45a53366669/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:01.006605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:01.006890 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:01.007194 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:01.037036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] MFA Rules not processed for user `42f05db0e91c4f38940ec45a53366669`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:01.038805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:01.119546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1384/2794] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:01.131667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:01.134304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:01.202511 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec28f1-7290-4ebb-b3f5-34c302380ab7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:01.206476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:01.207137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:01.207409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:01.207672 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:01.266243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:01.267730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:01.276863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1411/2795] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:05.967389 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:05.977422 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c1e13-071e-452f-bebc-335ec464dd1f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:05.981393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:05.981997 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:05.982277 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:05.982550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:05.992630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:05.994067 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8ffe-8cbf-4d65-b71f-beab5db23022 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:06.004242 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1385/2796] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:11.272863 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:15.999856 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=48 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:28.139776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:28.224964 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca48b3-6665-4774-b6a8-aa202314863c service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:28.232991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:28.233659 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:28.233916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:28.234187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] PATH_INFO: `/v3/users/53f9627e13204a688cfb9fca73ac4425` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:28.235550 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f9627e13204a688cfb9fca73ac4425)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:28.237544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:28.294175 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f9627e13204a688cfb9fca73ac4425'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.301994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f9627e13204a688cfb9fca73ac4425'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.302752 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f9627e13204a688cfb9fca73ac4425'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.318423 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f9627e13204a688cfb9fca73ac4425'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.327711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f9627e13204a688cfb9fca73ac4425'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.333932 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1412/2797] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:28 2022] DELETE /identity/v3/users/53f9627e13204a688cfb9fca73ac4425 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:28.338609 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:28.339132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:28.339251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:28.359288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:28.360492 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:28.424212 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1386/2798] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:28.438154 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:28.440253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:28.502711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86623-5c8f-41bc-8919-c1803b5d9028 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:28.506749 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:28.507396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:28.507667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:28.507935 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:28.517396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:28.518843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:28.528366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1413/2799] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:28.817100 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:28.818642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:28.833390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30fc21-2b3b-495c-a259-34a344f6bc8f None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:28.838365 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:28.839022 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:28.839303 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:28.839571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] PATH_INFO: `/v3/projects/20f88522579847c7ae4025746621a3d7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:28.840750 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f88522579847c7ae4025746621a3d7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:28.843257 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:28.893585 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f88522579847c7ae4025746621a3d7', 'request_id': 'req-2a52f768-4078-466a-ae49-1d03d56d8d53'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.899381 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f88522579847c7ae4025746621a3d7', 'request_id': 'req-2a52f768-4078-466a-ae49-1d03d56d8d53'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.908340 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f88522579847c7ae4025746621a3d7', 'request_id': 'req-2a52f768-4078-466a-ae49-1d03d56d8d53'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.920312 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] The token cache is being invalidate because project 20f88522579847c7ae4025746621a3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:28.920597 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.921309 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:28.922932 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1387/2800] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:28 2022] DELETE /identity/v3/projects/20f88522579847c7ae4025746621a3d7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:28.928908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:28.943899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5825b-c436-49b5-9dca-33ab9d18e084 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:28.966167 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:29.014234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.014868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.015188 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.015458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] PATH_INFO: `/v3/users/42f05db0e91c4f38940ec45a53366669` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.016679 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f05db0e91c4f38940ec45a53366669)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.018610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.078709 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f05db0e91c4f38940ec45a53366669'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.086381 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f05db0e91c4f38940ec45a53366669'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.087174 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f05db0e91c4f38940ec45a53366669'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.100447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f05db0e91c4f38940ec45a53366669'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.109664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-24592701-cb2d-455b-8896-47153fe321de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f05db0e91c4f38940ec45a53366669'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.115767 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1414/2801] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:28 2022] DELETE /identity/v3/users/42f05db0e91c4f38940ec45a53366669 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:29.408404 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.413738 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:29.424150 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52f768-4078-466a-ae49-1d03d56d8d53 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:29.482660 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.483254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.483550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.483801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] PATH_INFO: `/v3/projects/383a13ba16a7414e9a9f54d5be7a937e` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.485024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383a13ba16a7414e9a9f54d5be7a937e)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.486941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.528939 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:29.548471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383a13ba16a7414e9a9f54d5be7a937e', 'request_id': 'req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.556204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383a13ba16a7414e9a9f54d5be7a937e', 'request_id': 'req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.567317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383a13ba16a7414e9a9f54d5be7a937e', 'request_id': 'req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.582886 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] The token cache is being invalidate because project 383a13ba16a7414e9a9f54d5be7a937e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:29.583219 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.583938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-f9ab8b66-ed9e-46da-ae9e-3ed39a434ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:29.585550 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1388/2802] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:29 2022] DELETE /identity/v3/projects/383a13ba16a7414e9a9f54d5be7a937e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:29.590864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.591682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.591682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.613456 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:29.614668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:29.693989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1415/2803] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:29.703689 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.703980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.704254 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.726093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:29.727354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:29.736186 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1389/2804] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:29.740495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.751022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804e20-1156-4879-85a7-f6bf0ec67459 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:29.756262 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.756867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.757145 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.757415 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.758076 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:29.758378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.760130 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.772870 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1416/2805] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:29.777332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.788445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9ce9-3a64-4142-aaf5-3883e0514508 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:29.794695 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.795418 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.795686 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.795952 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.796606 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.798676 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.833433 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1390/2806] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:29 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:29.839329 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.852073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d47fa-4f7a-4c9a-abd2-09e45c3cbbdb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:29.879588 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:29.917341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.917959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.918233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.918501 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.919211 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.921253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.945817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1417/2807] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:29 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:07:29.950504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.962533 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7075e8-b598-4579-900a-8c66e0a3ef27 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:29.968083 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:29.968699 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:29.969102 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:29.969459 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:29.970136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:29.971822 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:29.982047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1391/2808] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:29.986920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:29.997375 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce283-5a12-4151-8a3f-2ef2d59b3696 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.002779 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.003410 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.003692 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.003961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.004602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.006205 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983fac2-decf-48db-b062-009412aad839 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.016519 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1418/2809] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:30.020603 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.032884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539a6a1-b55e-44bc-90cc-fd4128218f36 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.037756 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.038409 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.038687 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.038959 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] PATH_INFO: `/v3/projects/dffb173438af492891b49a5cb0db5e9d/users/318615fa9bb84e6da76e02a6c6584248/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.053943 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffb173438af492891b49a5cb0db5e9d, user_id=318615fa9bb84e6da76e02a6c6584248, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.056637 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.072397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1392/2810] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:30 2022] PUT /identity/v3/projects/dffb173438af492891b49a5cb0db5e9d/users/318615fa9bb84e6da76e02a6c6584248/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:30.083782 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.084068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.084340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.124771 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] MFA Rules not processed for user `318615fa9bb84e6da76e02a6c6584248`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:30.126010 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:30.202180 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1419/2811] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:30.207063 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.289761 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade9a37-ebcc-4083-b555-1fa9cc5770bf admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.296769 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.297378 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.297655 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.297931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.298567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.300515 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.332960 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1393/2812] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:30 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:30.336732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.348685 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c130a-3829-40c3-98ab-55d176ae0887 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.416169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.416791 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.417068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.417340 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.417985 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.420102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.445190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1420/2813] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:30 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:07:30.448985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.460234 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e990e-a453-4117-9a5e-a8a791597922 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.465781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.466430 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.466724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.467028 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.467723 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.469407 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.481217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1394/2814] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:30.485802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.496253 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e363e76-9b41-4f85-ab00-e971f9f6a1e8 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.501583 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.502220 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.502526 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.502800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] PATH_INFO: `/v3/projects/42f4badbe8a147eb854f4700956a1287/users/779fa42b87c743fbba0f3e287697d02f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.522378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f4badbe8a147eb854f4700956a1287, user_id=779fa42b87c743fbba0f3e287697d02f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.523875 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.539330 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1421/2815] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:30 2022] PUT /identity/v3/projects/42f4badbe8a147eb854f4700956a1287/users/779fa42b87c743fbba0f3e287697d02f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:30.543502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.553910 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be2ef-a90a-433a-be83-12393ff847d7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.574591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:30.614574 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.615210 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.615485 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.615752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.616377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.617976 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb88b3-1541-4885-a669-2acf2848964f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.627941 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1395/2816] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:30.632345 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.637959 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:30.644078 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53086450-1953-4a0d-ac42-53bd0a9199db admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.649806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.650480 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.650753 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.651044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] PATH_INFO: `/v3/projects/42f4badbe8a147eb854f4700956a1287/users/779fa42b87c743fbba0f3e287697d02f/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.652978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f4badbe8a147eb854f4700956a1287, user_id=779fa42b87c743fbba0f3e287697d02f, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.655746 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.665883 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1422/2817] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:30 2022] PUT /identity/v3/projects/42f4badbe8a147eb854f4700956a1287/users/779fa42b87c743fbba0f3e287697d02f/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:30.677301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.677591 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.677873 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.715805 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] MFA Rules not processed for user `779fa42b87c743fbba0f3e287697d02f`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:30.717470 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:30.793136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1396/2818] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:30.805409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.807688 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:30.873261 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d5d17-36af-42a7-8e30-4f7b59beb3b5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:30.877760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:30.878395 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:30.878667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:30.878936 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:30.888528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:30.889977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8336d-42ad-4de2-8666-1bec2fad66e5 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:30.899720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1423/2819] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:30.929394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:30.997086 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b6c6c-23c6-4839-a50e-87bc5c62bf67 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:31.002454 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:31.003106 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:31.003386 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:31.003653 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] PATH_INFO: `/v3/users/318615fa9bb84e6da76e02a6c6584248` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:31.004811 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=318615fa9bb84e6da76e02a6c6584248)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:31.006731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:31.057216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318615fa9bb84e6da76e02a6c6584248'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.058333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:31.067851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318615fa9bb84e6da76e02a6c6584248'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.077005 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318615fa9bb84e6da76e02a6c6584248'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.077617 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318615fa9bb84e6da76e02a6c6584248'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.090147 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318615fa9bb84e6da76e02a6c6584248'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.096590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1397/2820] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:30 2022] DELETE /identity/v3/users/318615fa9bb84e6da76e02a6c6584248 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:31.101506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:31.101839 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:31.102144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:31.121668 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:31.122908 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:31.183640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1424/2821] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:31.194157 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:31.195713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:31.266694 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8459105-785d-4711-8ccc-14e95da11dae admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:31.270821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:31.271457 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:31.271728 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:31.272016 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:31.282111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:31.283623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:31.293467 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1398/2822] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:31.577697 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:31.579232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:31.594972 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b1085-9028-4f3f-889f-a2094e849ccf None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:31.600610 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:31.601557 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:31.602039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:31.602473 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] PATH_INFO: `/v3/projects/dffb173438af492891b49a5cb0db5e9d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:31.603789 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffb173438af492891b49a5cb0db5e9d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:31.605595 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:31.659081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffb173438af492891b49a5cb0db5e9d', 'request_id': 'req-436cf383-7491-4b69-8aee-8fcd6f291b6d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.665816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffb173438af492891b49a5cb0db5e9d', 'request_id': 'req-436cf383-7491-4b69-8aee-8fcd6f291b6d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.672777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffb173438af492891b49a5cb0db5e9d', 'request_id': 'req-436cf383-7491-4b69-8aee-8fcd6f291b6d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.688804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] The token cache is being invalidate because project dffb173438af492891b49a5cb0db5e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:31.689088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.689701 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.691259 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1425/2823] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:31 2022] DELETE /identity/v3/projects/dffb173438af492891b49a5cb0db5e9d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:31.695419 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:31.705785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cb0c5-ceb6-4d2a-b8aa-dd688b9f9d06 service neutron] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:31.769929 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:31.770571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:31.770884 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:31.771166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] PATH_INFO: `/v3/users/779fa42b87c743fbba0f3e287697d02f` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:31.772336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=779fa42b87c743fbba0f3e287697d02f)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:31.774280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:31.832669 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779fa42b87c743fbba0f3e287697d02f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.840204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779fa42b87c743fbba0f3e287697d02f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.850226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779fa42b87c743fbba0f3e287697d02f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.850881 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779fa42b87c743fbba0f3e287697d02f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.856280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:31.864650 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-9436a722-0142-4141-a326-377f89d75a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779fa42b87c743fbba0f3e287697d02f'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:31.870980 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1399/2824] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:31 2022] DELETE /identity/v3/users/779fa42b87c743fbba0f3e287697d02f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:32.162658 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.164239 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:32.180235 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cf383-7491-4b69-8aee-8fcd6f291b6d admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.195087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:32.248831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.249477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.249756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.250022 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] PATH_INFO: `/v3/projects/42f4badbe8a147eb854f4700956a1287` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.251637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f4badbe8a147eb854f4700956a1287)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.253776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.306858 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f4badbe8a147eb854f4700956a1287', 'request_id': 'req-8e5e7add-2439-49bb-8493-a80b0f88c58b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:32.314041 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f4badbe8a147eb854f4700956a1287', 'request_id': 'req-8e5e7add-2439-49bb-8493-a80b0f88c58b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:32.320374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f4badbe8a147eb854f4700956a1287', 'request_id': 'req-8e5e7add-2439-49bb-8493-a80b0f88c58b'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:32.335272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] The token cache is being invalidate because project 42f4badbe8a147eb854f4700956a1287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:32.335554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:32.336222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8e5e7add-2439-49bb-8493-a80b0f88c58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:32.337777 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1426/2825] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:32 2022] DELETE /identity/v3/projects/42f4badbe8a147eb854f4700956a1287 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:32.342955 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.343278 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.343555 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.365673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:32.366825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:32.438093 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1400/2826] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:07:32.450447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.450928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.451432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.473509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:32.474739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:32.484153 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1427/2827] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:32.488810 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.499536 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde934b-b2f0-4f00-8a88-2da17909a79c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.505225 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.505827 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.506107 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.506379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.507038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:32.507353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.509024 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.521839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1401/2828] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:32.526394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.536312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e24e-0031-4347-9d34-0c873bbc55a6 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.542399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.543034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.543312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.543576 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.544229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.546144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.576400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1428/2829] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:32.581411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.591858 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdafff-78ba-4b66-8632-d5db3b3441ae admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.655799 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.656326 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.656640 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.656896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.657541 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.659605 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.685489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1402/2830] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:32.692491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.708055 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94306d4f-71d6-40a4-8f05-ec389e4c03a6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.713747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.714359 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.714640 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.714905 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.715585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.717212 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.730133 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1429/2831] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:32.734325 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.744875 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad819-e7bd-463f-8f10-e4ff94337362 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.749554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.750164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.750439 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.750797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.751483 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.753095 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80e0e3-5b2b-49f9-98a2-bf4358e7c6a6 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.763274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1403/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:32.767844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:32.783267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeac1f-8ad9-4787-9060-3c4880c1d634 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:32.789025 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:32.789711 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.789977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.790223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] PATH_INFO: `/v3/projects/31889d506f49476ba65a69bc180b2d8d/users/779e580a25ea404090bec89ca7d7ff4a/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.805946 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31889d506f49476ba65a69bc180b2d8d, user_id=779e580a25ea404090bec89ca7d7ff4a, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:32.808769 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:32.824304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1430/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:32 2022] PUT /identity/v3/projects/31889d506f49476ba65a69bc180b2d8d/users/779e580a25ea404090bec89ca7d7ff4a/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:32.840578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:32.840946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:32.841333 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:32.872283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] MFA Rules not processed for user `779e580a25ea404090bec89ca7d7ff4a`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:32.873490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:32.920076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:32.951041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1404/2834] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:32.956114 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.002420 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:33.029293 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d842a1-3345-4a4d-8822-a7b0b6e8366c admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.036113 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.036725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.037032 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.037307 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.037951 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.039916 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.069338 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1431/2835] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:32 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:33.073568 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.087721 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a0463-c075-4f84-819e-09edeba1de5c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.149891 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.150503 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.150777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.151043 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.151692 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.153662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.176393 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1405/2836] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:33 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:33.180567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.191591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ae87-8406-432f-b165-c5b979a7cd23 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.196687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.197289 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.197566 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.197837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.198490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.200127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.210976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1432/2837] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:33.215553 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.232582 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa21e-21e0-4825-acea-123c384e6137 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.240502 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.241162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.241437 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.241697 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] PATH_INFO: `/v3/projects/bc6c4fec195a48ca926d93b6e1964187/users/11e9864b94c24669829b0f12738a8a75/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.257530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6c4fec195a48ca926d93b6e1964187, user_id=11e9864b94c24669829b0f12738a8a75, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.260231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.275968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1406/2838] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:33 2022] PUT /identity/v3/projects/bc6c4fec195a48ca926d93b6e1964187/users/11e9864b94c24669829b0f12738a8a75/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:33.280084 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.290616 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd58dd9-71b7-40ec-a7b0-c17181370116 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.349027 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.349667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.349944 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.350221 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.350888 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.352517 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbcdd0-71ea-4d39-a2a5-0c7665cbd924 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.362682 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1433/2839] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:33.367940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.378763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bcf98-26ec-4f49-95cd-3e2b2e07626c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.383898 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.384552 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.384825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.385098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] PATH_INFO: `/v3/projects/bc6c4fec195a48ca926d93b6e1964187/users/11e9864b94c24669829b0f12738a8a75/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.387096 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6c4fec195a48ca926d93b6e1964187, user_id=11e9864b94c24669829b0f12738a8a75, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.389790 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.401127 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1407/2840] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:33 2022] PUT /identity/v3/projects/bc6c4fec195a48ca926d93b6e1964187/users/11e9864b94c24669829b0f12738a8a75/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:33.473127 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.473529 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.473810 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.504554 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] MFA Rules not processed for user `11e9864b94c24669829b0f12738a8a75`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:33.505871 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:33.602280 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1434/2841] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:33.616094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:33.619324 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:33.681612 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64502456-6947-4316-8773-1d8a853ac350 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:33.685362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:33.685968 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:33.686238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:33.687136 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:33.698530 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:33.700035 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:33.709684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1408/2842] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:34.050899 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.062301 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89983d-832b-4b25-bc76-1bc7392aafcb None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:34.067615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11e9864b94c24669829b0f12738a8a75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c4fec195a48ca926d93b6e1964187', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:34.068372 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.068687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.068967 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.069675 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:34.069977 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:34.071831 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:34.084187 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1435/2843] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:34.088455 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.097944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd00e-ac5a-4ed7-b6a0-393c47b08c64 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:34.103570 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11e9864b94c24669829b0f12738a8a75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c4fec195a48ca926d93b6e1964187', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:34.104171 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.104448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.104720 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.105390 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:34.107371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:34.136958 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1409/2844] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:34 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:34.141077 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.151915 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320bfd8-db8e-4e63-98a9-359b0a67d197 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:34.174292 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:34.214687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11e9864b94c24669829b0f12738a8a75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c4fec195a48ca926d93b6e1964187', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:34.215366 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.215645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.215913 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.216567 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:34.216866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:34.218484 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:34.232543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1436/2845] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:34.272747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.283244 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18023d27-8b31-4ada-a674-ae08908d82af tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:34.288572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11e9864b94c24669829b0f12738a8a75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc6c4fec195a48ca926d93b6e1964187', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:34.289198 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.289478 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.289775 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] PATH_INFO: `/v3/projects/1d5ff85088cf411ebbacce7d8590aaa7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.291036 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d5ff85088cf411ebbacce7d8590aaa7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:34.293238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:34.304925 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:34.345414 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5ff85088cf411ebbacce7d8590aaa7', 'request_id': 'req-eb97be88-2a6f-480c-994f-93065616ddd2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.352007 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5ff85088cf411ebbacce7d8590aaa7', 'request_id': 'req-eb97be88-2a6f-480c-994f-93065616ddd2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.359869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5ff85088cf411ebbacce7d8590aaa7', 'request_id': 'req-eb97be88-2a6f-480c-994f-93065616ddd2'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.372432 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] The token cache is being invalidate because project 1d5ff85088cf411ebbacce7d8590aaa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:34.372724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.373460 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-eb97be88-2a6f-480c-994f-93065616ddd2 tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.375129 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1410/2846] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:34 2022] DELETE /identity/v3/projects/1d5ff85088cf411ebbacce7d8590aaa7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:34.587441 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.589403 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:34.659142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1563e4c-306d-4c8f-83ae-b5945faa180f tempest-VolumeQuotaClassesTest-1534219999 tempest-VolumeQuotaClassesTest-1534219999-project-admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:34.723267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:34.723973 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.724244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.724513 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] PATH_INFO: `/v3/users/779e580a25ea404090bec89ca7d7ff4a` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.725761 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=779e580a25ea404090bec89ca7d7ff4a)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:34.727737 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:34.781498 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779e580a25ea404090bec89ca7d7ff4a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.788730 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779e580a25ea404090bec89ca7d7ff4a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.789408 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779e580a25ea404090bec89ca7d7ff4a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.797083 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:34.803445 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779e580a25ea404090bec89ca7d7ff4a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.815918 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779e580a25ea404090bec89ca7d7ff4a'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:34.823727 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1437/2847] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:34 2022] DELETE /identity/v3/users/779e580a25ea404090bec89ca7d7ff4a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:34.828341 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:34.828656 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:34.828924 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:34.849351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:34.850547 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:34.918835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1411/2848] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:34.931068 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:34.933431 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:35.003538 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a1cd8-1b19-4329-ab9d-e66b7b79e6fc admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:35.007878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:35.008528 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:35.008802 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:35.009071 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:35.018521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:35.020008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:35.030222 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1438/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:35.316768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:35.318524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:35.330768 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07953105-461f-4d4f-9aba-b3a384afbbf7 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:35.336135 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:35.336781 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:35.337061 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:35.337336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] PATH_INFO: `/v3/projects/31889d506f49476ba65a69bc180b2d8d` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:35.338501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=31889d506f49476ba65a69bc180b2d8d)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:35.340465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:35.391167 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31889d506f49476ba65a69bc180b2d8d', 'request_id': 'req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.399436 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31889d506f49476ba65a69bc180b2d8d', 'request_id': 'req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.409817 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31889d506f49476ba65a69bc180b2d8d', 'request_id': 'req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.421867 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] The token cache is being invalidate because project 31889d506f49476ba65a69bc180b2d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:35.422155 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.422809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.424351 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1412/2850] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:35 2022] DELETE /identity/v3/projects/31889d506f49476ba65a69bc180b2d8d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:35.428603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:35.438843 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f5312-70ed-4a0e-8cb6-3fed3e9c7520 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:35.501056 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:35.501828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:35.502178 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:35.502427 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] PATH_INFO: `/v3/users/11e9864b94c24669829b0f12738a8a75` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:35.504382 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e9864b94c24669829b0f12738a8a75)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:35.506991 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:35.565380 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e9864b94c24669829b0f12738a8a75'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.572889 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e9864b94c24669829b0f12738a8a75'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.573637 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e9864b94c24669829b0f12738a8a75'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.586837 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e9864b94c24669829b0f12738a8a75'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.596573 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-62a82cb7-c6e6-4533-8b4e-aebf544b0c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e9864b94c24669829b0f12738a8a75'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:35.603277 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1439/2851] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:35 2022] DELETE /identity/v3/users/11e9864b94c24669829b0f12738a8a75 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:35.876185 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:35.877621 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:35.894307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283a4-c7cd-49b2-990b-afc0a7b789dd admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:35.915683 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:35.958596 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:35.959280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:35.959554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:35.959818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] PATH_INFO: `/v3/projects/bc6c4fec195a48ca926d93b6e1964187` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:35.961073 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6c4fec195a48ca926d93b6e1964187)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:35.963087 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.012267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6c4fec195a48ca926d93b6e1964187', 'request_id': 'req-a5d0183b-87de-4aa1-b891-ee573c5057ad'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:36.017954 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6c4fec195a48ca926d93b6e1964187', 'request_id': 'req-a5d0183b-87de-4aa1-b891-ee573c5057ad'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:36.027772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6c4fec195a48ca926d93b6e1964187', 'request_id': 'req-a5d0183b-87de-4aa1-b891-ee573c5057ad'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:36.039428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] The token cache is being invalidate because project bc6c4fec195a48ca926d93b6e1964187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:36.039712 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:36.040338 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-a5d0183b-87de-4aa1-b891-ee573c5057ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:36.041836 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1413/2852] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:35 2022] DELETE /identity/v3/projects/bc6c4fec195a48ca926d93b6e1964187 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:36.047037 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.047347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.047627 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.062559 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:36.069337 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:36.071233 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:36.132693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1440/2853] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:36.141182 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.141465 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.141736 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.167448 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:36.169232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:36.180044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1414/2854] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:36.184650 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.195733 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00665-3dc4-49fa-9841-e9759fefd563 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.201930 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.202578 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.202854 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.203162 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.203853 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:36.204163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.205768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.218457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1441/2855] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:36.222869 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.232701 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc160dc-f0f2-4692-b73f-248529c40c6c None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.239216 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.240004 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.240368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.240638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.241546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.244522 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.283874 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1415/2856] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:36.288477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.299368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2973a0-62de-4104-be24-484cf0cbe732 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.356396 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.357081 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.357290 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.357530 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.358163 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.360125 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.388335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1442/2857] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:36.389565 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.400783 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc79e2-e0e8-46c6-94a1-0f6933269997 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.406532 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.407177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.407452 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.407716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.408354 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.409981 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.421802 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1416/2858] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:07:36.426168 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.437169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbafbef-c31f-4d93-96e1-73414f47b763 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.443045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.443646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.443920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.444186 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.444832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.446463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca66d-f28d-4a30-bd13-2b1c7ea136f2 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.458299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1443/2859] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:36.463307 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.476944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464ecdc-1f38-42cf-aa67-a458ca9c594d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.482184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.483179 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.483509 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.483877 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] PATH_INFO: `/v3/projects/fb9fe49b33c445428fa8dd6583836217/users/4a1a720deeb84d7fa3ec1a827eca675d/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.506821 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9fe49b33c445428fa8dd6583836217, user_id=4a1a720deeb84d7fa3ec1a827eca675d, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.509677 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.524861 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1417/2860] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:36 2022] PUT /identity/v3/projects/fb9fe49b33c445428fa8dd6583836217/users/4a1a720deeb84d7fa3ec1a827eca675d/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:07:36.535402 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.535687 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.536082 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.566089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] MFA Rules not processed for user `4a1a720deeb84d7fa3ec1a827eca675d`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:36.567593 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:36.643475 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1444/2861] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:36.648256 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.706876 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:36.724032 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3f05a-eea2-4ecd-af56-f86f09fd41f3 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.728905 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.729518 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.729797 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.730069 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.730706 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.732681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:36.761300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1418/2862] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:36.765515 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:36.771325 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:36.778684 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bed2d-a484-4323-a6f2-ac74c086d346 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:36.973756 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:36.974535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:36.975052 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:36.975449 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:36.976137 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:36.978039 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.003118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1445/2863] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:36 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:37.008082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.023937 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c20e3-4ec2-4b80-af93-ba25b37bedbb admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.030204 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.030825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.031137 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.031406 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.032040 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.033691 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.045191 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1419/2864] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:37.049527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.060747 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb0f9-7b9a-4924-9ac7-02046a9b27f5 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.066994 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.067739 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.068031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.068341 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] PATH_INFO: `/v3/projects/409cb983f90e4020a9f090d995b81baf/users/6e3764357ff146f6b0157c821bb21001/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.084045 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409cb983f90e4020a9f090d995b81baf, user_id=6e3764357ff146f6b0157c821bb21001, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.086683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.101463 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1446/2865] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:37 2022] PUT /identity/v3/projects/409cb983f90e4020a9f090d995b81baf/users/6e3764357ff146f6b0157c821bb21001/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:07:37.105517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.117445 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d271b5-4edc-450f-b1db-559d23cc776c admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.176394 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.177013 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.177317 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.177588 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.178229 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.179888 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d02d9d-5338-4845-b2bd-a9b9f79fa93f admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.190379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1420/2866] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:37.195040 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.205978 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927810-54d3-40f7-b60c-5149519617f9 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.211490 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.212142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.212418 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.212680 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] PATH_INFO: `/v3/projects/409cb983f90e4020a9f090d995b81baf/users/6e3764357ff146f6b0157c821bb21001/roles/e9f15fd6c7df4864ae7b25f72b495b90` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.214813 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409cb983f90e4020a9f090d995b81baf, user_id=6e3764357ff146f6b0157c821bb21001, role_id=e9f15fd6c7df4864ae7b25f72b495b90)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.217476 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.228474 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1447/2867] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:37 2022] PUT /identity/v3/projects/409cb983f90e4020a9f090d995b81baf/users/6e3764357ff146f6b0157c821bb21001/roles/e9f15fd6c7df4864ae7b25f72b495b90 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:37.240275 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.240567 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.240839 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.270972 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] MFA Rules not processed for user `6e3764357ff146f6b0157c821bb21001`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:37.272376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:37.345973 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1421/2868] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:37.358132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.360061 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:37.420047 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66f32b-9525-46ef-8697-b0e3f7af62c7 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.424191 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.424844 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.425101 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.425368 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.434628 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.436115 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.446065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1448/2869] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:37.534292 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:37.545428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095872dd-a3d3-476d-b559-8fba44eda5e1 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:37.550012 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:37.550643 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:37.550921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:37.551245 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:37.595936 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:37.612501 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:37.613947 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dde627-047c-43ca-82bf-589f28ceb0ce service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:37.623379 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1422/2870] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:37.957664 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:38.017003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:38.021438 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789578e-d356-40da-983b-0276d965eeb9 service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:38.027144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:38.027818 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:38.028089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:38.028351 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] PATH_INFO: `/v3/users/4a1a720deeb84d7fa3ec1a827eca675d` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:38.029477 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a1a720deeb84d7fa3ec1a827eca675d)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:38.031424 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:38.082226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a1a720deeb84d7fa3ec1a827eca675d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.089170 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a1a720deeb84d7fa3ec1a827eca675d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.089828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a1a720deeb84d7fa3ec1a827eca675d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.103806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a1a720deeb84d7fa3ec1a827eca675d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.116089 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a1a720deeb84d7fa3ec1a827eca675d'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.125373 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1449/2871] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:37 2022] DELETE /identity/v3/users/4a1a720deeb84d7fa3ec1a827eca675d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:38.131504 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:38.131791 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:38.132108 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:38.159221 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:38.160613 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:38.232902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1423/2872] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:38.244585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:38.246243 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:38.313265 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070bae7-370b-4201-b2db-15c8e042729f admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:38.317478 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:38.318095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:38.318391 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:38.318661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:38.328942 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:38.330400 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:38.340335 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1450/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:38.624940 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:38.626416 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:38.641716 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea497cb-62b4-4528-83cc-a56497cc3528 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:38.646999 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:38.647280 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:38.647579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:38.647850 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] PATH_INFO: `/v3/projects/fb9fe49b33c445428fa8dd6583836217` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:38.649336 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb9fe49b33c445428fa8dd6583836217)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:38.651251 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:38.699766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb9fe49b33c445428fa8dd6583836217', 'request_id': 'req-e4720e39-907e-4b08-9a61-6b7bb69759be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.705739 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb9fe49b33c445428fa8dd6583836217', 'request_id': 'req-e4720e39-907e-4b08-9a61-6b7bb69759be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.713816 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb9fe49b33c445428fa8dd6583836217', 'request_id': 'req-e4720e39-907e-4b08-9a61-6b7bb69759be'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.725831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] The token cache is being invalidate because project fb9fe49b33c445428fa8dd6583836217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:38.726120 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.727306 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.728931 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1424/2874] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:38 2022] DELETE /identity/v3/projects/fb9fe49b33c445428fa8dd6583836217 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:07:38.733118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:38.743683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51183fd5-26f5-4b15-86c9-e9b96d8eb2b3 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:38.815285 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:38.816097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:38.816447 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:38.816807 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] PATH_INFO: `/v3/users/6e3764357ff146f6b0157c821bb21001` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:38.818491 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3764357ff146f6b0157c821bb21001)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:38.821070 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:38.845549 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:38.873732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3764357ff146f6b0157c821bb21001'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.880673 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3764357ff146f6b0157c821bb21001'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.881334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3764357ff146f6b0157c821bb21001'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.895735 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3764357ff146f6b0157c821bb21001'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.907448 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3516ba95-6974-45fc-a8a5-0bf47b1efbfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3764357ff146f6b0157c821bb21001'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:38.913611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1451/2875] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:38 2022] DELETE /identity/v3/users/6e3764357ff146f6b0157c821bb21001 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:39.201912 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.203519 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:39.217897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720e39-907e-4b08-9a61-6b7bb69759be admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.279662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.280226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.280507 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.280786 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] PATH_INFO: `/v3/projects/409cb983f90e4020a9f090d995b81baf` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.282101 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] RBAC: Authorizing `identity:delete_project(project_id=409cb983f90e4020a9f090d995b81baf)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.284149 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.292383 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:39.335747 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409cb983f90e4020a9f090d995b81baf', 'request_id': 'req-84486dec-8413-4276-97bb-aaf35d94fcab'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:39.341260 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409cb983f90e4020a9f090d995b81baf', 'request_id': 'req-84486dec-8413-4276-97bb-aaf35d94fcab'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:39.351579 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409cb983f90e4020a9f090d995b81baf', 'request_id': 'req-84486dec-8413-4276-97bb-aaf35d94fcab'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:39.363329 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] The token cache is being invalidate because project 409cb983f90e4020a9f090d995b81baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:39.363616 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:39.364262 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-84486dec-8413-4276-97bb-aaf35d94fcab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:39.365785 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1425/2876] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:39 2022] DELETE /identity/v3/projects/409cb983f90e4020a9f090d995b81baf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:39.370920 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.371234 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.371506 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.391594 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:39.392800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:39.457323 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1452/2877] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:07:39.469571 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.469851 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.470132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.492267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:39.493622 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:39.502361 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1426/2878] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:39.506793 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.518266 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd437-18f3-48cb-b355-6a65ed90cffa None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.523303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.523989 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.524244 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.524510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.525161 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:39.525458 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.527087 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.540097 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1453/2879] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:39.545084 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.559946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43538240-95e5-402b-b56e-530ea6a49dcd None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.567529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.568359 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.568784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.569187 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.570267 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.572200 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.606749 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1427/2880] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:39 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:39.611654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.622409 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaeef-e631-4fbe-8b95-738a81949719 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.688468 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.689326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.689722 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.690095 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.691132 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.694102 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.718443 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1454/2881] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:39 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:39.724902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.736831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde91d1-d42c-46a0-adf0-48acc5e9f58d admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.742865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.743511 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.743784 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.744047 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.744684 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.746289 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.756539 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1428/2882] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:39.761023 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.771535 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cb7ea-3af6-4b54-996c-3bb70e5a7597 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.776548 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.777148 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.777432 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.777718 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.778364 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.780008 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43688e8-20e9-4758-8464-913898e89903 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.792798 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1455/2883] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:39.797154 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.808927 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fc4b-004e-4cf5-a9a6-2b711bf1f5a8 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:39.814741 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:39.815545 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.815823 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.816091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] PATH_INFO: `/v3/projects/5293de96b4c84d3bb4b8e34068c3371e/users/ce9c3b5391e84d7b9cb7d830cd5f4f6c/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.834097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5293de96b4c84d3bb4b8e34068c3371e, user_id=ce9c3b5391e84d7b9cb7d830cd5f4f6c, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:39.837041 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:39.853801 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1429/2884] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:39 2022] PUT /identity/v3/projects/5293de96b4c84d3bb4b8e34068c3371e/users/ce9c3b5391e84d7b9cb7d830cd5f4f6c/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:39.864652 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afee58-119c-4328-8043-20824edbd564 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:39.864961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afee58-119c-4328-8043-20824edbd564 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:39.865229 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afee58-119c-4328-8043-20824edbd564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:39.873805 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:39.895720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-76afee58-119c-4328-8043-20824edbd564 None None] MFA Rules not processed for user `ce9c3b5391e84d7b9cb7d830cd5f4f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:39.896928 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-76afee58-119c-4328-8043-20824edbd564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:39.980118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1456/2885] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:39.995190 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:39.997162 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:40.058112 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d1aac-5bcd-4111-b560-69dcefeb79a7 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:40.062332 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:40.062938 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:40.063247 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:40.063514 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:40.072114 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:40.073542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec4bee-9d9f-4add-a234-4eb64ddcebdf service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:40.082896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1430/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:47.599800 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afee58-119c-4328-8043-20824edbd564 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:47.664092 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afee58-119c-4328-8043-20824edbd564 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:47.670993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:47.671591 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:47.671867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:47.672136 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] PATH_INFO: `/v3/users/ce9c3b5391e84d7b9cb7d830cd5f4f6c` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:47.673352 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9c3b5391e84d7b9cb7d830cd5f4f6c)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:47.675304 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:47.737639 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9c3b5391e84d7b9cb7d830cd5f4f6c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:47.745043 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9c3b5391e84d7b9cb7d830cd5f4f6c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:47.745731 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9c3b5391e84d7b9cb7d830cd5f4f6c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:47.758123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9c3b5391e84d7b9cb7d830cd5f4f6c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:47.767622 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9c3b5391e84d7b9cb7d830cd5f4f6c'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:47.773623 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1457/2887] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:47 2022] DELETE /identity/v3/users/ce9c3b5391e84d7b9cb7d830cd5f4f6c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:47.778272 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:47.778766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:47.778868 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:47.798295 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:47.799546 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:47.831169 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:47.863638 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1431/2888] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:47.873804 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:47.875439 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:47.890943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:47.935112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcb7e6-8e79-481e-bb1c-16e62f2a93e6 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:47.939648 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:47.940252 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:47.940524 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:47.940785 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:47.949732 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:47.951217 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9b96c-a974-4431-baa9-6da6165ec117 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:47.960878 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1458/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:48.256635 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.258961 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:48.277946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6cb34-fcbd-4ec5-a051-75417ae40912 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.283441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.284091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.284371 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.284645 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] PATH_INFO: `/v3/projects/5293de96b4c84d3bb4b8e34068c3371e` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.285828 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5293de96b4c84d3bb4b8e34068c3371e)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.287853 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.340076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5293de96b4c84d3bb4b8e34068c3371e', 'request_id': 'req-8cb7d70b-29be-4e8d-9b12-125d7e556c83'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:48.345441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5293de96b4c84d3bb4b8e34068c3371e', 'request_id': 'req-8cb7d70b-29be-4e8d-9b12-125d7e556c83'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:48.355985 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5293de96b4c84d3bb4b8e34068c3371e', 'request_id': 'req-8cb7d70b-29be-4e8d-9b12-125d7e556c83'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:48.368053 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] The token cache is being invalidate because project 5293de96b4c84d3bb4b8e34068c3371e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:48.368344 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:48.369091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-8cb7d70b-29be-4e8d-9b12-125d7e556c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:48.370718 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1432/2890] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:48 2022] DELETE /identity/v3/projects/5293de96b4c84d3bb4b8e34068c3371e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:48.376009 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.376272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.376556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.397613 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:48.398880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:48.461066 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1459/2891] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:48.469111 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.469426 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.469733 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.491232 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:48.492442 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:48.500674 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1433/2892] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:48.504976 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.515288 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8a9e5-345d-4fe9-9661-b7e46c91b412 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.520118 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.520720 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.520993 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.521260 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] PATH_INFO: `/v3/domains` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.521906 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105787) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:48.522208 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.523845 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.537254 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1460/2893] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:48.541714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.551630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a16db-ce12-4b10-8788-f945f0be4e9e None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.556516 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.557130 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.557408 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.557681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] PATH_INFO: `/v3/projects` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.558357 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.560350 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.596835 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1434/2894] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:48 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:07:48.601241 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.612543 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252bbb6-5ec5-4736-aa44-b5a2539c656e admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.680998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.681705 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.681921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.682218 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] PATH_INFO: `/v3/users` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.682998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.685063 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.715694 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1461/2895] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:48 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:48.720276 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.731057 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd010488-49fb-492a-8c9e-e2e3f5e9663b admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.736950 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.737556 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.737831 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.738121 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.738743 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.740397 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.751089 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1435/2896] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:48.755103 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.765663 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e15a7-8767-4219-ae2a-0bb28ca9fde4 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.771429 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.772033 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.772312 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.772582 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.773232 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.774898 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a10eb-e3a4-4e09-a3b8-b2bd1b38e769 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.789012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1462/2897] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:48.793865 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.804623 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec84f80-1b79-48c7-b08c-7d45802b247e admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:48.810319 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:48.810979 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] REQUEST_METHOD: `PUT` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.811300 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.811576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] PATH_INFO: `/v3/projects/413877cae5214e78b40edcc8a896ba95/users/34aac11c7a0a4eeaa52c5690755310d7/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.826731 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=413877cae5214e78b40edcc8a896ba95, user_id=34aac11c7a0a4eeaa52c5690755310d7, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:48.829466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:48.844122 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1436/2898] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:48 2022] PUT /identity/v3/projects/413877cae5214e78b40edcc8a896ba95/users/34aac11c7a0a4eeaa52c5690755310d7/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:48.854457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f663-8094-481e-a88b-dc71a3520526 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:48.854736 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f663-8094-481e-a88b-dc71a3520526 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:48.854998 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f663-8094-481e-a88b-dc71a3520526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:48.883933 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-4dd8f663-8094-481e-a88b-dc71a3520526 None None] MFA Rules not processed for user `34aac11c7a0a4eeaa52c5690755310d7`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:48.885123 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-4dd8f663-8094-481e-a88b-dc71a3520526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:48.896708 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:48.963699 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1463/2899] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:48.976164 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:48.979184 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:49.053175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240550c-c7a2-4ad8-b6ac-13190687eb29 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:49.057787 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:49.058403 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:49.058681 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:49.058948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:49.068529 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:49.069964 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:49.073466 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:49.080305 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1437/2900] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:50.456961 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:50.457521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:50.457521 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:50.477880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] MFA Rules not processed for user `34aac11c7a0a4eeaa52c5690755310d7`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:50.479093 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:50.486817 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1464/2901] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:51.985493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:51.986921 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:52.058490 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25536765-a0ae-4b66-9b84-28e2e76b5f31 service cinder] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:52.065020 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:52.065633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:52.065907 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:52.066175 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:52.067904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:52.079495 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:52.081079 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:52.090727 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1438/2902] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:53.998617 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:54.000925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:54.065248 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3639c8f-ebe5-4657-b972-b780b15bfe07 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:54.068996 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:54.069806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:54.070185 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:54.070544 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:54.072868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:54.087827 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:54.089278 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:54.098880 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1465/2903] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:07:54.738930 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:54.740633 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:54.821500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e40dd-c6ec-46ad-8430-6d38d84bbee7 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:54.827471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:54.828085 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:54.828482 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:54.828763 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:54.830558 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:54.845133 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:54.845362 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ff915-bd43-478e-9e8b-90971920eef0 service nova] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:54.856343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1439/2904] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:00.030821 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:00.032884 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:00.051202 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9a2b6-08b2-436b-8156-101c0686f3e3 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:00.062806 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28f8f65dc678403081189e31ef5f8a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:00.063112 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:00.064003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:00.064003 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:00.070925 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:00.084482 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:00.086073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:00.097215 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1466/2905] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:04.415059 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dfd1-d63e-4b20-af3c-0c544914faff None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:04.415590 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dfd1-d63e-4b20-af3c-0c544914faff None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:04.415693 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dfd1-d63e-4b20-af3c-0c544914faff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:04.483031 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-1596dfd1-d63e-4b20-af3c-0c544914faff None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:04.484651 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-1596dfd1-d63e-4b20-af3c-0c544914faff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:04.523880 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=100 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:04.555782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1440/2906] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 01:08:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:04.615781 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:04.617399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:04.712571 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16423835-8369-4925-8a9d-b08b9a7e2ded service nova] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:04.718966 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:04.719630 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:04.719890 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:04.720203 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:04.728540 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:04.730124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:04.738316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1467/2907] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:05.300714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb809ac-acb0-4460-a02d-ef123e52ac59 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.301094 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb809ac-acb0-4460-a02d-ef123e52ac59 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.301494 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb809ac-acb0-4460-a02d-ef123e52ac59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.323343 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-feb809ac-acb0-4460-a02d-ef123e52ac59 None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:05.324542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-feb809ac-acb0-4460-a02d-ef123e52ac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.333177 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1441/2908] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:05.345611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.352275 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=100 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:05.357049 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8705b-67d5-4847-9d0a-369fab67c2a7 service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.363065 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.363612 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.363887 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.364156 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.372667 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.374242 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.382586 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1468/2909] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:05.630097 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91992-2605-4931-9094-d8becd76bbac None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.630620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91992-2605-4931-9094-d8becd76bbac None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.630620 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91992-2605-4931-9094-d8becd76bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.657377 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-abd91992-2605-4931-9094-d8becd76bbac None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:05.658572 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-abd91992-2605-4931-9094-d8becd76bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:05.666231 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1442/2910] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 30 01:08:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:05.678413 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:05.690143 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ed613-820a-4991-9e75-8e1cd90fd166 service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:05.696108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:05.696725 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:05.697116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:05.697394 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:05.705866 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:05.707500 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:05.715412 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1469/2911] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:06.058524 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88101191-470e-456c-b5e6-668ecae1568d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.058809 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88101191-470e-456c-b5e6-668ecae1568d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.059117 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88101191-470e-456c-b5e6-668ecae1568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.079453 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-88101191-470e-456c-b5e6-668ecae1568d None None] MFA Rules not processed for user `9aabe3099e2147e29be189622d916500`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:06.080641 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-88101191-470e-456c-b5e6-668ecae1568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:06.088722 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1443/2912] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 30 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:06.100921 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:06.112166 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5637-87a8-4625-93dd-ed7b152da6f4 service swift] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:06.118315 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b98ce88b48c34731a1eae85d23de837a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:06.118943 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:06.119247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:06.119509 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:06.128683 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:06.130190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3ada0-d99c-4af9-9023-fa9ab955b98d service swift] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:06.138556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1470/2913] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 30 01:08:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:08:08.430498 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88101191-470e-456c-b5e6-668ecae1568d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:08.499662 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88101191-470e-456c-b5e6-668ecae1568d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:08.504996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:08.505664 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.505965 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.506246 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] PATH_INFO: `/v3/users/34aac11c7a0a4eeaa52c5690755310d7` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.507471 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34aac11c7a0a4eeaa52c5690755310d7)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:08.509480 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:08.567630 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34aac11c7a0a4eeaa52c5690755310d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.576642 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34aac11c7a0a4eeaa52c5690755310d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.586376 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34aac11c7a0a4eeaa52c5690755310d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.587110 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34aac11c7a0a4eeaa52c5690755310d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.602489 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34aac11c7a0a4eeaa52c5690755310d7'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:08.608500 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1444/2914] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:08:08 2022] DELETE /identity/v3/users/34aac11c7a0a4eeaa52c5690755310d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:08.613399 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.613965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.613965 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.637816 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:08.639091 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.704867 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1471/2915] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:08.718818 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:08.720619 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.772449 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:08.796017 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bcc6b-d51a-4332-9bef-86fcb2303475 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:08.800353 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:08.800966 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.801211 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.801517 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.811044 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:08.812944 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cc0dc-e38c-4697-880e-cf0e45aea6db service neutron] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:08.822554 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1445/2916] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:09.001437 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.003088 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.015297 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c047d3-b753-4932-9907-342217ec7704 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.021044 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.021686 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.021962 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.022230 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] PATH_INFO: `/v3/projects/413877cae5214e78b40edcc8a896ba95` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.023768 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=413877cae5214e78b40edcc8a896ba95)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.025451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.084533 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413877cae5214e78b40edcc8a896ba95', 'request_id': 'req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.091457 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413877cae5214e78b40edcc8a896ba95', 'request_id': 'req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.097116 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413877cae5214e78b40edcc8a896ba95', 'request_id': 'req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.112326 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] The token cache is being invalidate because project 413877cae5214e78b40edcc8a896ba95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105787) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:08:09.112585 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.113267 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-3e04a668-84aa-426f-8cd3-df93f4e1a9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:08:09.114828 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1472/2917] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:08:08 2022] DELETE /identity/v3/projects/413877cae5214e78b40edcc8a896ba95 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:08:09.121038 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.121322 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.121602 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.148015 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.149618 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.219752 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1446/2918] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:09.230099 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.230361 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.230676 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.258318 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.260374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.275334 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1473/2919] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:09.282170 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.293946 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12ab44-9ac9-4272-8f8d-a86604eea0f8 None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.300411 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.301098 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.301428 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.301714 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] PATH_INFO: `/v3/domains` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.302352 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105788) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:08:09.302646 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.304368 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.323274 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1447/2920] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:09.328193 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.338886 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b66ae-5c67-4006-a8d9-f0b8738e86c1 None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.345797 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.346609 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.354012 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.354347 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] PATH_INFO: `/v3/projects` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.355147 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.357316 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.395755 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1474/2921] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:08:09.400283 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.414815 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb040157-352b-4634-a7b8-e3ae5b6b3771 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.520068 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.521056 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.521301 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.521788 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] PATH_INFO: `/v3/users` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.522996 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.526764 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.559148 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1448/2922] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:08:09.568223 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.590190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33ebf-4f2d-4470-98b6-571bf3f023be admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.597350 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.598272 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.598690 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.599152 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.600135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.602545 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.622231 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1475/2923] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.627990 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.645796 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ddca0-2e6a-4fc2-af15-bba32f0e8c77 admin admin] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.652217 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.652897 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.653203 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.653474 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.654128 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.655777 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ed93c-dd3e-43ce-a1f2-03fffc153d53 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.671152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1449/2924] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:08:09.678495 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.694126 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9c913-cf94-4eae-8a5e-284351f3e6fb admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.700956 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.701646 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] REQUEST_METHOD: `PUT` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.701922 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.702196 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] PATH_INFO: `/v3/projects/a16714c5d5764197a61b90b9ce3661ba/users/bd00ed08709940969d44e416358073eb/roles/fd7cb5f266b5465eb35b727b0d541b07` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.706689 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:09.721645 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16714c5d5764197a61b90b9ce3661ba, user_id=bd00ed08709940969d44e416358073eb, role_id=fd7cb5f266b5465eb35b727b0d541b07)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.724618 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.742311 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1476/2925] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 30 01:08:09 2022] PUT /identity/v3/projects/a16714c5d5764197a61b90b9ce3661ba/users/bd00ed08709940969d44e416358073eb/roles/fd7cb5f266b5465eb35b727b0d541b07 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:08:09.747627 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffe573-5c9d-4fa1-bfb9-e05956a6c59d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.747916 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffe573-5c9d-4fa1-bfb9-e05956a6c59d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.748226 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffe573-5c9d-4fa1-bfb9-e05956a6c59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.769152 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-e3ffe573-5c9d-4fa1-bfb9-e05956a6c59d None None] MFA Rules not processed for user `65d191ebc1aa4d0ebd90e4982cc77782`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:09.770531 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-e3ffe573-5c9d-4fa1-bfb9-e05956a6c59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.826872 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:09.847713 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1450/2926] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:09.862481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:09.864894 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:09.936303 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcb8f2-0a52-4747-9e71-95afaa1e7257 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:09.942451 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:09.943150 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:09.943425 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:09.943693 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:09.957661 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:09.960096 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.984615 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1477/2927] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:10.744320 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557b83-2d91-4215-b1dc-f18f75fd60db None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.744919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557b83-2d91-4215-b1dc-f18f75fd60db None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.744919 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557b83-2d91-4215-b1dc-f18f75fd60db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.775928 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-4b557b83-2d91-4215-b1dc-f18f75fd60db None None] MFA Rules not processed for user `bd00ed08709940969d44e416358073eb`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:10.777173 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-4b557b83-2d91-4215-b1dc-f18f75fd60db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:10.849091 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1451/2928] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:10.860938 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:10.862481 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:10.927762 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a920ec3-0db7-442e-a5e6-69466b9a5da7 service neutron] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:10.931599 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87af1d19163043ffaa3412487446b02f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:10.932225 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:10.932499 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:10.932780 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:10.946948 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:10.949013 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:10.959510 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1478/2929] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:12.378709 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] REQUEST_METHOD: `POST` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:12.379293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:12.379293 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:12.405825 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.auth.core [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] MFA Rules not processed for user `bd00ed08709940969d44e416358073eb`. Rule list: `[]` (Enabled: `True`). {{(pid=105788) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:12.407238 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:12.420782 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1452/2930] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:08:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:12.702536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:12.704603 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:12.723378 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65e098-810c-43a8-8901-810858dbe73d service cinder] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:12.727776 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3acf83ba003e4e02a5ddd87c2714792d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:12.728497 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:12.728864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:12.729299 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:12.731459 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:12.743374 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:12.745034 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df33c5-3664-4e7b-9a01-9b6b992061be service neutron] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:12.757654 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1479/2931] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:22.399493 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=42 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:22.750760 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:57.318076 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:57.319772 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:57.397673 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adc49f-8932-4d52-b9bf-02c18f13b32d None None] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:57.403124 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:57.403766 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:57.404045 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:57.404311 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:57.406132 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:57.420158 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:57.421864 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:57.433724 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1453/2932] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:07.428434 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=32 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:12.348947 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] REQUEST_METHOD: `POST` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.349556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.349556 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:12.370868 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.auth.core [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] MFA Rules not processed for user `bd00ed08709940969d44e416358073eb`. Rule list: `[]` (Enabled: `True`). {{(pid=105787) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:12.372139 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:12.381110 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1480/2933] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 30 01:09:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:22.364794 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=3 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:06.538862 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:06.540512 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:06.553902 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7df0d-38e0-48f5-9b09-5078485d5413 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:06.559667 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:06.560288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] REQUEST_METHOD: `GET` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:06.560542 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:06.560848 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:06.563166 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.fernet_utils [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105788) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:06.575860 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:06.577639 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:06.591602 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:06.599740 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1454/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:06.603385 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab49044-c64b-43bc-9733-a0021b4c795a None None] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:06.608536 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:06.609169 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:06.609632 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:06.610073 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] PATH_INFO: `/v3/users/bd00ed08709940969d44e416358073eb` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:06.612031 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd00ed08709940969d44e416358073eb)` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:06.614190 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:06.671825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd00ed08709940969d44e416358073eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:06.678527 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd00ed08709940969d44e416358073eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:06.679226 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd00ed08709940969d44e416358073eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:06.691247 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd00ed08709940969d44e416358073eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:06.699611 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.notifications [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd00ed08709940969d44e416358073eb'} {{(pid=105787) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:06.705472 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1481/2935] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:06 2022] DELETE /identity/v3/users/bd00ed08709940969d44e416358073eb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:06.910578 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] Authenticating user token {{(pid=105788) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:06.923183 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039ba73-4990-4f62-a96f-d5d7c1defbb0 service placement] Validating token access rules against request {{(pid=105788) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:06.983082 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d191ebc1aa4d0ebd90e4982cc77782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa5e798ada384a4f94fc4eb7a83dd47f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105788) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:06.983734 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:06.984021 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:06.984288 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] PATH_INFO: `/v3/projects/a16714c5d5764197a61b90b9ce3661ba` {{(pid=105788) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:06.985387 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16714c5d5764197a61b90b9ce3661ba)` {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:06.987441 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] RBAC: Authorization granted {{(pid=105788) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:07.043948 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16714c5d5764197a61b90b9ce3661ba', 'request_id': 'req-5bd11c9c-f44e-4c47-b99e-266cace0fb24'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:07.050671 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16714c5d5764197a61b90b9ce3661ba', 'request_id': 'req-5bd11c9c-f44e-4c47-b99e-266cace0fb24'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:07.061904 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16714c5d5764197a61b90b9ce3661ba', 'request_id': 'req-5bd11c9c-f44e-4c47-b99e-266cace0fb24'} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:07.074608 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] The token cache is being invalidate because project a16714c5d5764197a61b90b9ce3661ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105788) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:07.074896 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:07.075576 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG keystone.notifications [None req-5bd11c9c-f44e-4c47-b99e-266cace0fb24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105788) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:07.077126 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: [pid: 105788|app: 0|req: 1455/2936] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:06 2022] DELETE /identity/v3/projects/a16714c5d5764197a61b90b9ce3661ba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:16.702238 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:17.071586 dv-cinder-815746-8-iscsi devstack@keystone.service[105788]: DEBUG dbcounter [-] [105788] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105788) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:02.290108 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Authenticating user token {{(pid=105787) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:02.293832 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:02.487987 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a69a3-eac5-40d3-9498-1d462cf54344 admin admin] Validating token access rules against request {{(pid=105787) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:02.571864 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22b14f894354d4197834889e75489a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '920fb02470584601a6dfea23df009f85', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105787) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:02.573124 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] REQUEST_METHOD: `GET` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:02.573631 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] SCRIPT_NAME: `/identity` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:02.574142 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105787) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:02.576995 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.fernet_utils [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105787) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:02.596144 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:02.598135 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f938cb0-cf14-4adc-aa08-5c8377e52f54 service placement] RBAC: Authorization granted {{(pid=105787) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:02.610825 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: [pid: 105787|app: 0|req: 1482/2937] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:12.604882 dv-cinder-815746-8-iscsi devstack@keystone.service[105787]: DEBUG dbcounter [-] [105787] Writing DB stats keystone:SELECT=89 {{(pid=105787) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}